We have updated our Privacy Policy and Privacy Options

Got It

How Can You Avoid the Most Common Testing Pitfalls? Brooks Bell Experts Share Their Insights


Woman at laptop

For a few companies, data is part of their DNA and experimentation is a way of life. For most others, testing is an ongoing process of encouraging buy-in, generating fresh test ideas, learning from losses, and troubleshooting technical problems. Our experts have identified key hurdles that companies face when trying to ramp up their testing programs, and they share here how to avoid them.

circle-headshot“Some companies value quantity over quality. Instead, they should slow down and focus on strategy.” Tweet_this

-Bryan Prehodka, Director of Project Management at Brooks Bell

Bryan Prehodka, director of project management, sees testing teams run into two problems. The first is that they don’t spend enough time on strategy. They want quick wins through quick tests, and any learnings they gain aren’t integrated well into their next test ideas.

The second issue is that they don’t take the time to QA properly. For example, a company may run a test and realize only later that they’re not capturing the metrics they needed. Not only does this hurt their bust rate, it impacts learnings and future tests. Something like this could be avoided through data validation before the test is pushed live.

suzi 2016“For programs in the early stage, it can be difficult to get people to take notice of testing, participate in it, support it—and not just try to poke holes in it.” Tweet_this

-Suzi Tripp, Director of Client Management at Brooks Bell

Sometimes the biggest challenge is just getting a testing program off the ground. Our director of client management, Suzi Tripp, acknowledged that larger buy-in from stakeholders can be difficult.

One approach is to help them understand that testing may not give you all the insights into your customers right off the bat, she said, “but it does give you actionable insights that you can utilize to make more money, get more members, or whatever your goal is.”

She notes that new programs that have initial success with testing need to be solid in the data. “Because of the success … the program gets pounded with all these questions. You have to be ready to be bulletproof in what you’re putting out there.”

mike-2016“If you don’t have good strategy, your learnings are going to be all over the place.” Tweet_this

-Mike Adams, Director of Optimization Engineering at Brooks Bell

Culture problems were also noted by our director of optimization engineering, Mike Adams. A common challenge, he said, is not having a good direction for your testing program. “In a lot of the larger companies, it can be a big political system. They’re hearing different things from different people and are just trying to make people happy with testing.”

Before a testing program can succeed, stakeholders must be aligned. Successful programs are built on a foundation of data, something acknowledged by another team member.

claire-2016“Some companies aren’t leveraging the vast amounts of data they have to inform their testing strategy. They’re still going with their gut, which can be more risky.” Tweet_this

-Claire Schmitt, Director of Optimization Consulting at Brooks Bell

From a consulting perspective, common challenges involve metrics, test duration, data, and process—or any combination of the four. It’s different for every company, says Claire Schmitt, director of optimization consulting, but she finds that many companies struggle with the same core issues:

  • Disagreeing over KPIs
  • Not running tests for the correct amount of time
  • Not being truly data driven (either not using data to make testing decisions, or not being sure how to use it)
  • Struggling with the testing process

Many companies dealing with process problem lack resources on the development side. One way to solve that is by hiring an agency like Brooks Bell, Claire says. “For those that don’t have the capacity to run tests internally, we enable them to get tests out the door. For those who are able to get tests out the door, we augment their team and help them increase testing velocity.”

What are some testing pitfalls you’ve encountered? How have you dealt with them?

Brooks Bell helps top brands profit from A/B testing, through end-to-end testing, personalization, and optimization services. We work with clients to effectively leverage data, creating a better understanding of customer segments and leading to more relevant digital customer experiences while maximizing ROI for optimization programs. Find out more about our services.