We have updated our Privacy Policy and Privacy Options

Got It

5 Fundamental Benefits of a Testing Repository


Increasing testing velocity is exciting, but managing the deluge of results and insights is a serious challenge. Unfortunately, many organizations fail to address the problem directly, meaning it falls on a single person managing the testing program. The responsibility of aggregating results shouldn’t fall on one person. That’s why many testing teams develop a central repository to store and share information.

Using a repository, or knowledge base, allows teams to house, tag and otherwise organize testing information and learnings—and access essential information whenever it is needed.

1. Eliminate the Single Point of Failure

The all-too-common alternative places the responsibility—and burden—of storing this knowledge on one person, and typically on that person’s memory.  Testing teams, though usually autonomous, are often disjointed. They borrow designers or developers from other departments and, because of this, have trouble communicating. Training new team members is a challenge for the same reason. Some members of the team only work on testing part time, so they may not be up to date with the latest developments. As a result, information is forgotten, insights are lost, and time is wasted with repeated tests. It’s a common mistake—and one we have learned from here at Brooks Bell, too.

Put simply: Organizations can’t afford to rely on a system with a single point of failure.

2. Gain Broader Insights

Repository Post_0In addition, keeping a repository can help gather larger metrics when looking at a program as a whole. Testing teams are able to organize the tests in their knowledge base by section or topic, and can gather data about a specific section of the site or a specific product or topic from the large numbers of tests they run. This makes it easy for large organizations running hundreds of tests per year to look at the rate of wins, losses, and flat tests to easily calculate overall trends.

3. Increase Accountability

When results start to roll in, it’s easy to get distracted jumping from test to test. Without establishing a knowledge base that can be referenced, testing teams can lose sight of the work they are putting into each test. In some respects, this is a good thing; it’s a sign that processes are falling into place. But it also has dangerous implications for the ROI of the testing program. Moreover, a repository of results makes regular performance audits possible—something that is particularly important for new programs or programs that outsource key tasks.

4. Lay a Foundation for Growth

Testing programs in any organization should be living, breathing entities. For this to be possible, the repository must be kept up to date. The gold standard in the industry is to have a designated historian or librarian within the testing team; a person who is held accountable for keeping test knowledge up to date and accurate. Some teams even create a dedicated position to accomplish this important task—often known as an internal “Testing Evangelist.”

5. Standardize Processes

For smaller teams and organizations that don’t have a dedicated testing team, the luxury of having a dedicated knowledge base curator may not be realistic. Fortunately, it is still possible to track this critical information and build a repository using a spreadsheet stored in SharePoint or some other shared server.

The spreadsheet should include spaces to record, at a minimum:

  • Campaign and Variation Name
  • Test Owner or Creator
  • Test Launch Date
  • Test Status
  • Section of the Site Affected
  • Results from the Test
  • Revenue Impact of the Test

To be sure that your testing program information is kept up to date and can be easily accessed and understood by new team members, its important to have standards for entry, and a balance of information. A formally designed and carefully managed knowledge base is the best way to achieve this.

By creating a functional, flexible repository of testing information, you guarantee that testing ideation continues to move forward, new team members are oriented to goals quickly, and the entire process can become more reliable and efficient. It improves testing today and ensures it will continue to improve into the future.

Have you implemented or utilized a test repository in your organization? Share your experience in the comments below!