We have updated our Privacy Policy and Privacy Options

Got It

What Leeroy Jenkins Can Teach Us About Testing

Share

There’s a fine line between recklessness and ambitiousness. In the case of Leeroy Jenkins, the viral memetastic World of Warcraft video that took the internet by storm 7 years ago, it was a little bit of both. In case you are unfamiliar with the legend of Leeroy Jenkins watch the video below and then let Wikipedia give you a quick overview:

The video was released by the World of Warcraft player guild “PALS FOR LIFE”. It features a group of players discussing a detailed battle strategy for the next encounter while one of their party members, Leeroy, is away from his computer. Their plan is ruined when Leeroy returns and, ignorant of the strategy, immediately charges headlong into battle shouting his own name in a stylized battle cry. His companions rush to help, but Leeroy’s actions ruin the meticulous plan, and all of the group members are killed.

Leeroy was both ambitious and reckless in his attempt to tackle the Rookery ahead of his guild in the Upper Blackrock Spire, known for its large quantity of dragon spawn. We can learn a lot from Leeroy when it comes to testing.

We spend a lot of time talking about planning your test roadmap, but sometimes in an attempt to cover all of our potential risk, we are left standing around computing probabilities and “what ifs.” Lately we have seen companies delaying their testing programs to make sure every possible scenario is covered. But this is the wrong approach when launching a testing program. We need to be a bit more like Leeroy Jenkins.

No matter how long you plan, new complications always pop up when launching the first test. These can include technical problems with your testing tool implementation, traffic not being correctly split, segments failing to track separately, etc. They can also include internal process issues like legal approval, timing release cycles, and the basic complications that go with getting ideas approved up the decision ladder.

The goal is to get your testing program launched. At Brooks Bell we start every testing program with a quick test launched within 14 days. Launching this test allows us to troubleshoot and optimize the testing process in a real working environment.

In order to launch a test this quickly a few things need to happen:

Data Sharing

Without a free flowing exchange of data surrounding the page or pages being optimized, it is very difficult to strategize the most appropriate approach to this initial test. Many times, the access to traffic and conversion data is held up by bureaucracy hurdles, so we try to put that process in motion as soon as possible.

Commitment to the Goal

The primary goal of the initial test is to gain experience and confidence in successfully launching a test. The secondary goal is to gain learnings towards optimization of conversion points. It’s paramount that all parties involved agree to this goal. While it is great to always see a lift, if you lose sight of the primary goal, you may start creating tests that require more design and development resources.  Those types of tests may move the needle but you should save them for later.

Commitment to Agressive Timelines

You can only move as fast as the slowest person so it is important to make every attempt to set clear expectations when discussing timelines and approvals. Because this initial test tends to be very basic and diagnostic in nature, approvals historically happen faster than normal.

Here’s an example of an initial test that we ran for one of our clients that accomplished our goals:

Our client has a well-trafficked content portal with a goal to increase page views from the featured stories module. Our initial test idea was to add thumbnails to the secondary stories to see if increased eye appeal would lead to an increase in engagement. As we started to set up the test we ran into an unexpected snag: We knew that Test&Target could not host the new image thumbnails we created but when we asked our client to host them for our test we found out that images couldn’t be uploaded until the next release cycle—a date that fell outside of our test plan timeframe. The success came from this new learning of our client’s release cycle schedule. A detail that we didn’t think to ask and one that our client didn’t think would affect our test strategy. This knowledge has allowed us to plan and time our future testing around their release cycle schedule.

Leeroy Jenkins ran into the Rookery and quickly found out the dragon spawn was… let’s say… a complication. As his guild gets decimated he famously utters the line, “At least I have chicken.” You may not move the needle in your conversion rate with your initial test, but the learnings you gain is invaluable to your testing program in the future.

At least you have chicken.

Categories