On October 4, we were treated to 2012’s first Presidential Debate. And to the average viewer, it was a barrage of numbers and data: 700 billion, 1 trillion, 250 million, 600, 1,250. Numbers and data are a great way to back up your ideas, but numbers without context are useless.
Let me preempt any protests from political zealots by saying that politics has no place in data. Data doesn’t care whether you stand on the right side of the aisle, or the left. Nor does it care if your tie is red or blue. That’s why we love data.
As the candidates were throwing out all of these data points, it got me thinking. Okay, you have a plan to shave $250B off of the deficit. The audience probably doesn’t know exactly how much that deficit is. So are you saying that you can shave off 50% of the deficit? Maybe it’s 25%? Maybe it’s only 1%? Where is the context? Without that, we don’t know the significance of the plan, or where it will leave us, deficit-wise.
You can save American families $1,250 a year? Great! But wait, is the American family three people in a household, or is it five? If $1,250 is based off a five-person family, what does that mean for the three-person family? How about the seven-person family? Having context here would help people find the data to be relatable, or at the very least allow them to do the math for their specific situation.
When you report data results from your site, you need context. Say you made changes to the membership signup funnel on your site, and it resulted in 100 new memberships. That’s great, right? Well, how much traffic was needed to drive those 100 new signups? Was it only 150 visitors, 150,000 or was it 1.5MM? This context is important to get buy-in from higher-ups for more testing, to inform your next steps and to track your data effectively.
Does your report show that fear- vs. fact-based subject lines increase open rates by 75%? Fantastic! But what’s your list size? Is it five people, or five million? That context is important.
Reporting data without context is useless. Sure, it can make your wins look stronger and your losses seem less significant, but that’s not a clean way to test.
What do you think? Do the reports you see on a weekly basis have enough context? Do your own reports have enough, or are you just trying to take the attention off of a lost test?