We have updated our Privacy Policy and Privacy Options

Got It

Why We Love Testing: Customers Rule


How much time do you spend debating about design and what you think the customers prefer?  Surely the snazzy page with the great images, color scheme, etc. is what the customer wants to see.  Unless you test, you’ll never know for sure.

I have a simple example. In a past project I had worked on, we created a simple registration page built to test against our control for our paid search efforts.  The main goal of the exercise was to decrease cost per registration.  The CPR was very high and my hypothesis was that there were too many distractions, the conversion button was on the next page and below the fold, etc. and it just seemed like it was too much effort to get a new user to register.

This particular test landing page was… ugly.  No graphics.  Standard font.  Just a simple registration form, value proposition in bullets, and action item above the fold.  At the time, I couldn’t get the resources to create a more professional looking page.  (Can’t get the resources you need–  Sound familiar in your organization?)

We split the traffic between the control site page, landing page A with registration form on left, and landing page B with registration box in the center.  Within days, we were seeing a drop in CPR by more than half!  Every penny counts these days and that’s significant.  I showed the results to the creative director and he had a hard time believing it.  Was it possible that this ugly page could beat out a page that we spent so much time designing and developing?  It’s hard to argue with numbers.  I calculated some simple $ impact scenarios short term and long.  After that, it didn’t take long to get resources to start building nicer pages.

The moral of the story is this:

  1. Test and let the customers decide what they want.  You can soooo want them to like your design or concept, but at the end of the day, you gotta let them decide what they want to see.
  2. It’s tough to get resources allocated to your projects unless you have numbers to show.  Not only is it numbers, but you have to put it in simple $ signs AND you have to sell it.  You can be the greatest statistician/analyst/marketer/scientist/etc. and have the most detailed analysis possible… but if you can’t boil it down to simple business terms and impact, you might as well have spent your time doing something else.

Even if you feel like you’re great at understanding your customers, if you aren’t testing and iterating, you’ll never know how much money you could be leaving on the table!