I spoke at SES NY (now called ClickZ Live NY) about a year ago. Here's a post I wrote which summed up the talk:
I’m often asked how did we get ramped up to doing 3–5 tests a week. I have to admit, I’m impressed that we’ve gotten this far, but we still have room to grow.
In short, it comes down to 3 things to get you there:
- Aligning your optimization strategies with your business goals
- Establishing the right framework and processes
I’ll break it down.
Align your optimization strategies with your business goals
It may seem obvious, but there’s often the impulse to just start testing everything everywhere. You’re enthusiastic and want to show ROI immediately, but you need a strategy. What are your company’s goals? Most often there are revenue goals you’re looking to meet and there are business strategies you need to tap into to help get you there. Is there messaging you can experiment with to support the business strategies? Creative expression? Layout? Functionality? Answering these questions can help guide you.
Establish the right framework and processes
It’s no question that if your company isn’t doing testing yet, it’s going to take a little time to shift the mindset; it’s a culture change. Depending on how open teams are to change and taking your business to the next level, will determine how quickly you can implement a program. You’ll need:
- Operating model alignment: You’ll have to get buy in across your organization. It might mean asking other teams to support your efforts or ask permission to take on some other tasks like designing pages.
- Process design: It’s going to depend on your organization how you want to work. I’ll elaborate more in the following paragraph.
- Change impact and readiness: How does adding testing affect other teams?
- Organizational design: Do you keep it all in one group or do you share responsibilities with other teams?
- End user training: Does the team have the skills they need?
If you’re doing it all yourself, you don’t have to work about processes as much. However, if you’re working across divisions or departments it’s best to have roles, responsibilities, and processes established and agreed upon. You’ll need to secure:
- Web strategist
- Optimization manager
- Campaign enablement
- Analytics support
- Web production
- Project management
- Web development (for more sophisticated testing)
- Demand and media generation (to drive traffic to your site)
- Creative development
- User experience expertise
- Platform architecture
Execute your tests
Now you have your strategies, a slew of test ideas which support your strategies and business goals, you have your team in place, now you have to execute. How do you know what to test first? The first rule of thumb is analyze which pages on your site yield the highest conversion, have the highest volume of traffic, or demonstrate the biggest area of opportunity. We’ve set up scoring models to help us with prioritization. As the team grows, you have to manage lots of tests getting ready to launch. Potentially all at the same time.
Our scoring model is simple. What type of test is it: Content, layout, functionality, or creative? What’s the level of effort: Low, medium, or high? What is the potential revenue lift: Low, medium, or high? The test type score is based on previous quarter’s successes. Have an analyst help you with that. The level of effort is judged by the person doing the enablement of the test which might include development. And the potential revenue impact is also evaluated by an analyst.
The optimization manager should develop their strategy and then come up with individual tests to support those strategies. Then methodically develop test charters or plans which outline the goals of the test, justification, what page(s) the test should be run on. It’s good to have a hypothesis, but be cautious on including that in your test charter because it might sway you to want certain results.
Set up your test using a solution like Adobe Target, run it 3–5 weeks (depending on traffic and conversion rates), and evaluate the success through the reports in the tool or if you want to dig deeper, use something like Adobe Analtyics. Use what you learn to feed your next test idea and keep iterating.
Watch your conversions increase
The more comfortable you get with testing, the quicker you can iterate on tests and reap the rewards of increased conversions. Happy testing!
Provide feedback on how you operate your optimization program.