5 Test Ideas to Learn More About Holiday Shoppers

November 12, 2015

David DeFranza

This year, Black Friday will look more like a month-long shopping journey than a one-day extravaganza. Ecommerce, of course will play a huge role in this holiday shopping season and mobile, more than ever, will represent an important channel for both research and conversion.

While the prospect of such a protracted high-volume shopping period may be daunting, there is an upside—a month or more of holiday shopping traffic provides ample opportunity for testing. Testing during the holiday can help you drive increases in revenue, of course, but it can generate insights into customer behavior applicable to the entire year, too.

To accomplish this, it’s critical that tests are designed intentionally, with a clear hypothesis and opportunity to learn regardless of the result. It’s not easy, but there are a few obvious approaches to try. These five will get you started:

1. Order Volume vs. Order Size

When it comes to ecommerce, the perennial challenge is balancing the number of orders with the size of each order. While not directly correlated, improving one often seems to come at the expense of the other. Through testing, it’s possible to determine the sweet spot between the two and identify exactly which side your customers fall on.

What to test: One interesting way to test this idea is to encourage additions to the cart or encouraging movement toward checkout. When a shopper adds an item to the cart in the first variation, for example, you could redirect them directly into the shopping bag for review. In the second variation, shoppers would be redirected back to a category grid page with a message prompting continued shopping.

What you’ll learn: Testing this behavior helps you determine whether your customers are interested in purchasing a single, specific item, or if they are looking to buy several products from your brand. Perhaps more importantly, it helps you learn which behavior is more responsive to influence. This helps refine KPIs and can inform many additional strategies moving forward.

2. Long vs. Short Path to Purchase

When a shopper lands on your site, is he or she eager to complete a specific purchase, or would a more conventional funnel be appropriate? This question is particularly relevant during the holiday season, when shoppers bounce site to site, often making several purchases across individual sessions.

What to test: Typically, product description pages feature a prominent “add to cart” CTA button. To test motivation to purchase, add a “quick checkout” CTA—which would allow the shopper to move directly from a product description page into the checkout funnel—to one variation.

What you’ll learn: The most obvious insight is, whether, given the opportunity, shoppers will convert at a high rate when presented with a clear opportunity to do so. Digging into the results will be helpful here as well. Of those that click the “quick checkout” CTA, for example, how many proceed directly through the funnel to complete purchase and how many bounce back or open a cart for review? Understanding this behavior will direct refining tests moving forward. This is also a particularly interesting test when launched across both desktop and mobile channels.

3. Browsers vs. Buyers

The reality of online retail is that it has, for many people, replaced traditional window-shopping. A significant portion of traffic is made up of shoppers conducting research, killing time, or browsing out of curiosity. These browsing shoppers are not without value, however; such behavior builds brand awareness and affinity and can influence later purchase.

What to test: Many ecommerce sites feature “quick view” options on category grid pages. To test the browsing behavior against the buying behavior, consider a variation that emphasizes the quick view CTA, another that replaces it with an “add to cart” CTA, and a third that features an “add to wish list” CTA.

What you’ll learn: Comparing the use of these CTA to final conversion in all three conditions will help you understand whether shoppers are intent on browsing—even if they default to using the cart for research.

4. New vs. Returning Shoppers

During the holiday season, many shoppers make several purchases across many sessions. The needs of these shoppers on subsequent visits may be very different than those of completely new shoppers. This creates an excellent opportunity for testing.

What to test: For returning visitors, testing a feature banner that emphasizes ongoing sales and promotions in one variation, most popular items in one variation, and “great when purchased with”—linked to previously browsed or purchased items—in a second variation. For new visitors, a feature banner that emphasizes ongoing sales and promotions in one variation, most popular items in the second can provide a point of comparison.

What you’ll learn: The primary insight from this test is whether new and returning visitors respond differently to promotion. Once this difference is established, it can inspire series of tests to develop a more nuanced understanding of the needs and preferences of each group.

5. Shipping Savers vs. Shopping Procrastinators

Free shipping is the classic incentive used to encourage purchase. It’s a powerful tool that can have a dramatic impact on both total orders and average order value. But during the holiday season, saving on shipping is the sole motivation for shoppers.

What to test: In one variation, emphasize a free shipping offer, making it clear shoppers can save by placing an order of a certain value. In the second variation, emphasize shipping is available to guarantee an item arrives before a specific date.

What you’ll learn: Saving on shipping costs is great, but can shoppers also be motivated by an assurance the item will arrive on time? Knowing this can influence shipping promotions around the year, once verified through additional testing, too.

Many retailers may feel it’s too late to optimize their sites for this shopping season, but thanks to the constant expansion of this period, there’s still plenty of opportunity to launch tests, increase revenue, and learn about customers, too.

—

Brooks Bell helps top brands profit from A/B testing, through end-to-end testing, personalization, and optimization services. We work with clients to effectively leverage data, creating a better understanding of customer segments and leading to more relevant digital customer experiences while maximizing ROI for optimization programs. Find out more about our services.