How Committed to Experimentation Are You, Really?

This post was written by Naoshi Yamauchi the chief performance officer (CPO) at Brooks Bell.

Amazon, Google, LinkedIn, Bing, Intuit—all amazing companies committed to experimentation. They are staffed with dozens of smart people and empowered by optimization systems that let their customers guide what they build. One executive at Amazon I talked to recently, for example, claimed they test down to the “pixel.”

I’ve had the opportunity to work with and research many organizations that run the range from just getting an experimentation program started to those that are and have been fully committed to experimentation as a critical part of their business. There are lots of factors that make one program better than others. At the root of it all sits culture  —the top-down belief that experimentation can drive business growth. While most major companies are executing experimentation, my observation is that the majority of these companies are not truly committed.

Here’s the question: Is your company committed to experimentation?

Think about kids involved in extracurricular activities. Parents block off practice schedules and games on our calendars but not all events are treated equally.

What Commitment Looks Like

When we’re committed, we make sure we plan resources and time so the kids get where they need be. Unless it’s an emergency or a really big deal, other activities will not trump their practices or games.

From the parental perspective, it’s obvious that the kids get to their practices and games. It’s important to their development and growth.

For a company committed to experimentation like LinkedIn, using data, testing, and other methods to make decisions is not only a rule, but a rule without exceptions. Experimentation, I have learned through discussions with program stakeholders there, is part of the DNA and a key driver of how their business evolves.

LinkedIn ensures experimentation is supported at every level through important initiatives and practices. For example:

1. Each product team has the skillset, resources, process, and platform to quickly execute tests.
2. A robust infrastructure and process has been built to make it quick and easy to run a test.
3. Development is integrated into experimentation so implementing a winner is often as simple as flipping the switch.

Suffering From a Lack of Commitment

When parents lack commitment, they begin to rationalize participation in activities. If no conflicting activities arise, for example, and if we can find transportation for them, our kids will probably go to practices and games.

From the parental perspective, we tend to think “OMG, getting the kids to practices or games is such a sacrifice and a monumental effort.”

If this kind of rationalization and resistance to experimentation sounds like your company, rest assured that you’re not alone. When an organization suffers from a lack of commitment to experimentation, it’s common for:

1. The “lead” to be doing most of the work. Because there are no dedicated resources, launching experiments is slow and difficult.
2. Experiments are often deprioritized and postponed because too many more “urgent” features are in the queue.
3. Every time an experiment launches, it feels like it has taken a monumental effort.
4. When there is an insight, it’s a Herculean feat to implement those learnings into an upcoming release cycle.

Take a step back and think about the comparisons above and it might help you see how your company really thinks about experimentation. After speaking with people from hundreds of experimentation programs, I know that the majority of companies fall into the latter, uncommitted group. Every person—and every business—faces choices and priorities. Companies like Google, LinkedIn, Amazon, and others made a commitment to experimentation years ago and it has helped them grow into the businesses we know today. But if you’re not in this group, it’s not too late. It’s just time to answer the question: How committed is your company to experimentation?

Naoshi Yamauchi is the chief performance officer (CPO) at Brooks Bell. In this role, he leads the analytics and engineering teams, oversees ideation and strategy, and refines processes and methodologies. Naoshi is focused on increasing win rate, pushing velocity, and deepening learning through testing for all Brooks Bell clients. He received an MBA in marketing and innovation management from North Carolina State University. 

—

Brooks Bell helps top brands profit from A/B testing, through end-to-end testing, personalization, and optimization services. We work with clients to effectively leverage data, creating a better understanding of customer segments and leading to more relevant digital customer experiences while maximizing ROI for optimization programs. Find out more about our services.