We have updated our Privacy Policy and Privacy Options

Got It

Why I like Losing Tests and Why You Should Too


I realize the headline to this post may sound strange considering Brooks Bell is a company focused on strategizing and executing tests that win and make money for our clients. I assure you, however, that I am not a pessimist.

I like losing tests.

Don’t get me wrong, this strange affection ends at “like.” I don’t love losing them. But from a pure testing standpoint I like losing tests just as much as I like winning them. Here are 5 reasons why you should like losing too.

Man with Loser sign

image: Flickr user: Jugbo under Creative Commons

1. Knowing what doesn’t work gives you just as much information as knowing what works.
No one likes to lose but every scientist knows losing a test yields just as much benefit in learning as winning a test. This is where the strategizing of smart tests is important. If you can’t pinpoint which element caused the test to win, you probably varied your test cells too much. The same is true when losing tests. Keep that in mind when you build out your test matrix.

2. Losing a test can sometimes reveal other “hidden” problems.
We are just as guilty of this as you are. When a test is winning, we go through our normal process to validate the numbers, but rarely do we obsess over them. It’s natural. Everyone has the intention to run a flawless test devoid of any data issues or tracking code misfires. But when that test you are sure will win, doesn’t, it typically warrants a second and third look into the data. In the end you may still have a losing test. In some cases with this second look, however, you may uncover an issue that affects reporting across the board.

3. Losing a test proves sometimes best practices don’t always win.
If I make a statement like “You can’t always depend on best practices to win in every situation” everyone will agree with me without hesitation. But if I ask “What should we do first to optimize the page?” you will most likely say “apply best practices first.” Losing a test like this can remind us that what works for some won’t work for everyone.

4. Losing prevents complacency.
Complacency leads to comfort and that ends up feeling satiated. Sure, you may have specific lift goals to hit with a campaign, but hitting goals early without a loss can lead to less risk-taking, deep analysis, and out-of-box strategy. Losing every once in awhile prevents that complacency.

5. Losing is healthy.
Like the old saying goes: “you win, you lose some.” The Loss Rate industry average is 25%. We don’t win 100% of the time and you won’t either. Obsessing over a perfect record is an exercise in futility and getting sick over it is a waste of time and energy. The attitude you should convey to your team is one where a loss is merely a smaller step towards optimization.

That’s why I like losing tests. Now’s your turn.

Do you have an optimization culture that embraces losses? Do you have other examples? Let us know in the comments.