What Every CMO Needs to Know About Testing

January 23, 2014

David DeFranza

In the increasingly data-driven world of marketing, smart decisions rely less on the experience of a senior executive and more on smart analysis. But that doesn’t mean a marketing team can excel without strategic direction. As traditionally campaign-focused teams begin to embrace testing, optimization, analytics and other methodologies, it’s critical they receive appropriate support from leadership.

So what do CMOs need to know about testing to provide this guidance? Essentially, there are three important areas—based on the RPV framework outlined by Clayton M. Christensen—senior leadership must consider to enable their teams to succeed with testing.

cmo_testing

Resources

From the beginning, testing requires some new resources. A testing tool, of course, must be implemented. Other software tools—including a separate analytics package, click or eye tracking, and a tag management system—may also be needed. This will require an investment both for the purchase of these platforms and also for implementation.

Beyond these obvious infrastructure investments, it’s important to consider the time your team will have to invest for the proper execution of a testing program. Initially, this time investment may be fairly minimal. But as testing expands and variations become more complex, the time—and team—required will increase.

A huge benefit Brooks Bell provides clients is a complete testing team including an analyst, a UX designer, a JavaScript engineer, and a project manager—and this is an excellent model when considering the skill and staffing requirements for your own program.

Process

For those interested in a quick overview of how testing works, there are many resources available. Most CMOs, however, don’t need to worry about the details of setting up campaigns, creating segments, or calculating significance.

Instead, upper management must ensure the testing program is aligned with the goals of the business. By providing a framework for creating useful hypotheses and prioritizing test ideas in relation to KPIs, CMOs can establish functional guidelines for their team to work within.

Values

Typically, values are thought of as ethical guidelines. While this is true, the term takes a broader meaning in this case. Specifically, organizational values define the way in which teams make decisions and establish priorities. “Clear, consistent, and broadly understood values,” Christensen explains, “define what an organization [can and] cannot do.”

Initiating a testing program often requires a shift in organizational values. Foremost is the freedom to try and fail—something that is particularly important in the early learning stages of testing. It’s important, during this stage, to focus on learning from each test, iterating ideas, and working towards the first big win.

It can also be difficult for some organizations to transition from experience or intuition-based strategy to data-driven strategy. Instead of letting opinions dictate strategy, it’s important to transform them into test ideas—then measure the effectiveness of each idea against one another.

Moving to a testing and optimization-based strategy can be challenging for teams accustomed to developing campaigns through assumptions and experience. CMOs looking to start a testing program, however, can maximize the chance for success and speed transition by focusing on the resources, processes, and values required to build a sustainable optimization culture.