We have updated our Privacy Policy and Privacy Options

Got It

Successful Testing Strategy: Determining When to Swing for the Fences


You may be familiar with the baseball term “swing for the fences.”  It’s the concept of swinging at the ball as hard as you can with the intention of hitting a home run.  The problem with this approach is that you are more likely to miss.  BUT, if you hit the ball, you know it’s out of the park.  A true “go big or go home” scenario.

Brooks Bell conversion experts often use this expression as it relates to testing.  Testing strategy can be approached in two different ways:

1. Element Tests

2. Complete Redesign Tests

Element Tests proceed in a clean, methodical fashion.  These are the singles and doubles of baseball – gradually making their way to home plate.  Changes from the control are isolated so that their exact impact is known.

The upside is that win, loss or tie – you’ve identified knowledge that is applicable throughout the testing roadmap, which is valuable information.  The downside is that this approach can be time consuming and lifts are often small per test, though they can collaboratively make a big impact over time.

Complete redesign tests take a “swing for the fences” approach.  After completing a control assessment, the control is given a complete overhaul that includes layout, design and copy changes.

The great thing about these tests is that it is a facelift for an out-of-date or underperforming control.  Not to mention that it’s more fun, right?  The downside is that since so many changes have been made to the page, there is no way to determine exactly what is (and isn’t) working.

So, what method is best?  Well, obviously the one that leads to winning.  I’m really asking the question of getting to the win via singles, doubles, triples and stolen bases vs. knocking it out of the park in a grand fashion.

As a Client Manager, I have clients on both sides of the fence.

Element tests that are executed in a one-after-the-other fashion with incremental lifts along the way can seem rather monotonous.  I like them because we can isolate exactly what works, implement it on the new control and move on to optimizing the rest of the elements on the page.  It’s a constant flow of learnings and it’s a bit more conservative.

On the other hand, some clients prefer to focus on larger tests, like complete redesigns.  On an emotional level, there’s just something about rolling out a redesign test that feels different than an element test.  I like the excitement that surrounds these types of tests.  More blood, sweat and tears go into it and you know something big is going to happen, be it a big win – or a big loss.

Is there really one right answer?  At Brooks Bell, we constantly debate which method to use during every test ideation.  And the truth is that it varies, so there is no right or wrong answer.  What’s imperative is that testing is done in a disciplined approach that allows you to reap a long-term benefit.

If a redesign is where you want to start, go for it!  Ideally, you can create multiple redesign challengers to test against the control.  Structure them differently so that there is clear differentiation between each cell so that you can learn something.  Perform an A/B split test to determine your winner.

How many baseball games have you seen won with home runs alone?  Win it like the pros using a combination of both methods!  Once you’ve determined the layout that works best, continue to do clean, methodical element tests to optimize the page.

When used together with a disciplined approach, these are two great methods that can lead to big learnings, big conversion rate lifts, and more revenue – the grand slam of testing.

Brought to you by Suzi Tripp and Naoshi Yamauchi