DATA-DRIVEN CMO is an ongoing series on the Brooks Bell Blog that focuses on topics for the modern day data-driven CMO.
My boys (8 and 6) were playing LEGO Star Wars on the Wii the other day having a grand ol’ time hacking Stormtroopers with lightsabers, when all of a sudden they got stuck on a level that they couldn’t get past. After 2 or 3 game restarts, they still couldn’t solve the board. I watched intently to see how they would overcome this challenge. Would they stay focused and figure it out? Would they try a new path to see if it’s any better? Would they learn from their mistakes? What happened next was remarkable.
Instead of battling through the level, they took their characters back to the cantina, where they customized their characters with new weapons, accessories and outfits. In essence, they took a more gratifying approach to their game time, rather than doing the hard work of figuring out the puzzle.
We marketers fall into this trap with conversion optimization. When we get stuck on an optimization program that yields flat results, we’re tempted to turn to the more gratifying approach. We test outfits and accessories—not new paths or kill moves. After all, it’s way more gratifying to test button designs, rather than psychology-driven messaging, isn’t it?
A common mistake marketers make is to turn to multivariate testing, and test a bunch of elements on a page at once. They may have the traffic volume, the right testing tool setup and the right data capture to determine a high-performing recipe, but that’s nothing more than learning which color lightsaber looks better with your Padawan robe. At the end of the day, you get something that looks good but isn’t necessarily advancing to the next level.
This is why we recommend A/B testing before multivariate. There is a place for fine-tuning with multivariate tests, but doing it before understanding how to move the needle within your path is a diversion and an avoidance of the issue. Figure out how to beat the level then worry about how to dress up your Jedi.