Taking an iterative approach to testing and optimization—allowing each carefully focused experiment to build on the results of those that came before—is the best way to develop a deep understanding of consumer behavior. With each successive experiment, you website becomes easier to use and more effective. And these changes are stable, repeatable, and extendable.
But sometimes, slow continuous improvement doesn’t produce the results that are expected, let alone demanded. For a number of reasons a rigorous, restrained approach to optimization may not meet the needs of your testing program or organization. In these cases, it makes sense to abandon a “small ball” strategy—characterized by a constant stream of small, incremental wins—and instead “swing for the fences.”
So how do you know it’s time to test a radical change? A “swing for the fences” approach could be a good option when:
You have limited time or traffic
If you have months to test and a lot of traffic to split between a dozen variations, it’s easy to keep tests clean and volume high. More often, however, results are needed quickly due to an impending feature launch, marketing campaign, shopping season or some other event. And for many sites, low traffic—or a low rate of conversion—requires tests to run for weeks or even months. In these cases, testing a dramatic change can help make the most of limited test volume.
You have a big goal
Testing programs often have a mandate to prove their value—especially when new or newly expanded. This can lead to huge ROI goals within a fairly short timeframe. While learning is always a desirable outcome from a long testing roadmap, scoring a big win to earn support and gain buy in early may be critical. Typically, big wins demand big changes—the kind that can muddy individual learning.
You have additional data
Testing teams shouldn’t work in isolation, limited only to the data captured by a testing tool and web analytics platform. Instead, data from consumer insights, customer service, marketing research, and other groups should be used to develop test ideas and strategies. And sometimes, this data points to a big change. A/B testing is a great way, for example, to triangulate the data and prototype solutions to the problems gained through usability tests.
You’ve already tested extensively
If you’ve spent months or years testing a website or funnel, chances are you’ve reached an optimization maximum—a point at which the page in question can’t be significantly improved. There’s always a chance, however, that this is only true for the page within a certain context and that all your tests to this point have missed a valuable group of customers. Identifying—not to mention addressing—the needs of this new group can be very difficult through small changes. Therefore, if results are plateauing, it might be a good time to try something radically different.
Babe Ruth is often credited with saying you can’t “let the fear of striking out hold you back.” Ultimately, incremental testing focused on developing clear, clean insight is the most sustainable way to optimize a website. Taking a rigorous approach generates the greatest insight and provides the best platform for future testing. But sometimes, chasing a big win requires a bolder approach.