We have updated our Privacy Policy and Privacy Options

Got It

How Does Testing Fit Into Agile Development Cycles?


If you have spent any time in IT or developer meetings, you’ve probably witnessed a discussion—if not a debate—over the merits of a waterfall methodology versus an agile approach. Indeed, the two perspectives on project management are bound to radically different views of process, design, and organization—and inspire intense opinion from those that have a preference.

The more traditional approach—the waterfall method—is a sequential design process that has its roots in manufacturing and construction industries. At the outset of a waterfall project, a detailed description of the requirements is laid out. This requirements outline dictates the course of subsequent stages which include design, development, QA testing, launch, and maintenance. The defining characteristic of this methodology is that each phase is executed individually and the project does not move on until the work in the phase has been completed.

The agile methodology was developed as an alternative to the rigid, linear process dictated by the waterfall approach. Instead of outlining a complete set of requirements at the outset of a project, the agile methodology utilizes user research and a product backlog or queue to define the next steps. This backlog is regularly reprioritized based on the changing needs of a business. Once an item has been selected, a dedicated, multidisciplinary team focuses exclusively on the task, working in a time-boxed “sprint” until the feature is finished. At the end of each sprint, the backlog is revaluated before the next sprint begins. The important distinction here is that each sprint represents one small component of a larger project—and development moves in an iterative and incremental way.

Obviously, the waterfall method made a lot of sense in the days of shrink-wrapped software that was physically published, packaged, and mailed to stores and customers. But in the world of web apps and software as a service, the rapid, iterative improvements enabled by an agile approach are much more appropriate. And this goes for marketing, too. The days of massive campaigns built around expensive ad buys are fading. Instead, marketers are using an agile approach to create and manage more targeted, relevant campaigns in real time.

Agile and Testing

Testing is of fundamental importance to the agile process. To illustrate this point, May Petry, HP’s VP of digital marketing, explained, “Agile development enables [HP] to test, learn, iterate, and execute much quicker.” The problem is that rapid development cycles and a demand for complete focus from developers can conflict with the needs—both in terms of run times and resources—of testing.

So how does testing fit into an agile development schedule? There are three basic models:

1. Create Parallel Backlogs

In what is likely the ideal scenario, an independent testing team works in parallel to the development team—and the two maintain related but distinct backlogs. Though it is critical the two teams remain aligned, the prioritization of the backlogs can differ in this arrangement. The advantage is that testing can happen continuously, providing constant feedback to the development teams through a stream of user data. This approach requires more resources and a deeper investment in testing—a hallmark of advanced testing organizations but a clear challenge for those launching or nurturing new programs.

2. Bookend Sprints

An alternative is to bookend sprints, launching tests during the backlog prioritization and project review phases. This could potentially allow a testing team to utilize development resources during the transition period between sprints. It opens an opportunity to use testing to help prioritize the backlog or to optimize new features. For programs with limited development resources but dedicated analysts, this is a great approach because the test can run and results can be analyzed during a sprint—with development for the next test occurring between sprint efforts.

3. Integrate Backlogs

The third approach integrates the testing and development backlog. In this case, each test will be designed, developed, launched, and analyzed as a sprint. For small organizations with limited resources, this may be the only reasonable option but there are obvious disadvantages. When testing must compete with feature development and bug fixes, velocity will be slow and there’s always a risk it will be pushed aside completely. Testing programs using this approach must make an effort to accurately estimate the potential impact of tests and communicate their importance during prioritization discussions.

Even though the agile philosophy is built around testing and data-driven development, finding the time and resources to actually execute tests can be a challenge. By understanding the various ways testing can work with and within an agile development cycle, however, it’s possible to manage the needs of both programs.

Brooks Bell helps top brands profit from A/B testing, through end-to-end testing, personalization, and optimization services. We work with clients to effectively leverage data, creating a better understanding of customer segments and leading to more relevant digital customer experiences while maximizing ROI for optimization programs. Find out more about our services.