Agenda
Tuesday
May 7th
Wednesday
May 8th
Thursday
May 9th
9:00 AM - 10:00 AM
Yoga at The Umstead (Optional)

Enjoy a class of yoga at The Umstead Hotel & Spa to recharge and ready for 3 days of stimulating conversation!  This class has limited spots and will be available with sessions to sign up for.  (Attendees only)

1:00 PM - 4:00 PM
Brooks Bell Trainings

Basics of Testing Analytics
1:00PM – 1:50PM

Whether you’re new to testing or you need a refresher, the Basics of Testing Analytics training is for you. Brooks Bell’s analytics team will go over everything from strategic analysis to experiment design to interpreting test results. Seats are limited to 25 and are available on a first-come, first-served basis.

Launch Better Tests with Brooks Bell’s Ideation Methodology
2:00PM – 2:50PM

Coming up with ideas for testing can be a stressful undertaking–especially when you want to make sure that those ideas are strategic, data-driven, grounded in behavioral science, meaningful and successful. Join Suzi Tripp, Brooks Bell Sr. Director – Innovative Solutions as she walks you through how Brooks Bell teams use a balance of technology and methodology to create winning test ideas. Seats are limited to 25 and are available on a first-come, first-served basis.

How to Plan and Launch Profitable Personalization Campaigns
3:00PM – 3:50PM

Struggling to make personalization work for your company? You’re not alone. Join this workshop to learn major considerations to keep in mind when developing your personalization strategy, how to best identify your customers and campaign opportunities, whether you should deliver your campaigns via rule-based targeting or algorithmic targeting, and more! Seats are limited to 25 and are available on a first-come, first-served basis.

1:00 PM - 5:00 PM
Brooks Bell Office Hours & Sponsor Demos

Grab a 30-minute slot to discuss testing program concerns or to see a product demo with Brooks Bell or one of the event sponsors.

6:00 PM - 10:00 PM
Opening Night at The Umstead

6:00pm Cocktails and networking
6:30pm Speed Networking
7:30pm Dinner
8:30pm Cocktails and networking

8:00 AM - 9:30 AM
Breakfast & Opening Keynote
9:30 AM - 10:45 AM
Conversation #1 Topics

Using Advanced Statistics to Scale your program

Moderator: Jesús León, American Express

Description: Let’s expand our toolkit beyond A/B testing box. This conversation will delve into how smarter designs of experiments and more advanced statistical techniques, can help to scale-up an experimentation program, increase insight output and solve for specific real-life business needs (low volumes, effects of interactions, etc.).

Program Management: Tactics, tools, troubles and triumphs

Moderator: Scott MacMillan, Fidelity

Description: When it comes to managing a decentralized testing program, the devil is in the details. Join this session to talk through the ins and outs of testing program management from avoiding test collisions to capturing and sharing testing insights.

Testing for Lifetime Value: How to ensure your experiments are helping people take action for the right reason

Moderator: Eric Allen, Ancestry.com

Description: Sometimes a test can offer up great short-term conversions, but it creates a poor user experience and, potentially, long-term losses. This session will dive into how to use testing to: learn the value of each point of the funnel, build loyalty and improve the user experience for positive long-term gains.

Omni-Measurement: Measuring the full impact of digital testing

Moderator: Chris Chapo, Gap Inc.

Description: As experimentation programs mature, they begin to expand beyond just measuring site conversions and AOV. Brands like GAP are beginning to experiment around profitability and how it’s impacted by shipping speed and personalized fulfillment. This conversation will discuss some of the applications and techniques outside of the traditional measurements, and how they are shaping the future of testing.

Testing Connected Cross-Channel Experiences

Moderator: Mark Jodlowski, American Family Insurance

Description: It’s already a challenge to deliver a consistent message across the customer journey; adding testing to the mix can seem impossible. Join this conversation to address these challenges and discuss potential solutions to test into a better experience between all channels.

The Role of AI and ML in Personalization

Moderator: Jason Hickey, Adobe Target

Description: When it comes to how to best implement and involve machine learning and artificial intelligence in your personalization campaigns, there tends to be more questions than answers. Join this session to discuss how to get the right results with machine learning, what information you need to input to get the desired output, and the roles of the various parties from data scientists to the executive sponsor.

10:45 AM - 11:15 AM
Morning Networking Break

A short break between conversations to network with other attendees or swing by Brooks Bell experts or any sponsors to answer any questions you might have.

11:15 AM - 12:30 PM
Conversation #2 Topics

Influencing without authority: How to advocate for testing in a federated environment

Moderator: Jeremy Yoches, Capital One

Description: Whether your testing environment is geographically distributed, federated, or both, with testing governance, the struggle is real. Join this session to talk through how to best influence teams working in different offices and time zones, how to codify testing standards and how to encourage collaboration and share learnings to build off each others’ successes.

Using Behavioral Economics to Optimize your Funnels

Moderator: Artem Petakov, Noom

Description: Join this session to discuss the role of behavioral economics in funnel optimization. We’ll discuss tactics like positive friction, using anticipation to improve conversation rate, the relationship between behavior and rewards and how to use ego to drive conversions.

Statistical Rigor: What is the right framework for testing?

Moderator: Dipesh Patel, Home Depot

Description: There’s a lot of debate in the testing world about statistical rigor. If you have an opinion, this is the session for you. In this session we’ll nerd out on sequential analysis, controlling for false positives, analyzing the impact when you’re running multiple tests in different parts of the funnel, how much you can count on out-of-the-box analytics from your testing tools, and more.

Leveling Up: How to properly mature your organization through the implementation of a CDP/DMP

Moderator: Will Chamberlin, BlackRock

Description:Everyone is trying to get a better understanding of their customers and prospects–and there’s a lot of technology leading the charge. Whether you are implementing a customer data platform (CDP) or a data management platform (DMP), or both, the first step is setting realistic expectations about these “silver bullets.” In this conversation we won’t focus on the actual platforms, but rather the goal they are trying to deliver and the framework for best maturing your company to a better understanding of your customers.

Personalization at Scale

Moderator: Matt Gardner, Chegg.com

Description: To implement personalization at scale, the technical and process implications are daunting. How do you ensure that your systems (testing, CMS, back-end) are connected so that you serve up the right experiences to the right people? What strategy and processes should exist to scale personalization? Join this conversation to discuss.

How Your Traditional Reporting Is Hiding the Ugly Truth About your Actual Customer Experience

Moderator: Matt Schaub, USAA

Description: There’s nothing worse than when you peel back the layers of your traditional reports and find out that your customer experience isn’t all Rainbows and Unicorns. Let’s talk about how to change your mindset away from old school reporting and discuss new and non-traditional ways to analyze and quantify the customer journey, particularly how to quantifying friction and struggle and increase the likelihood of a conversion. Let’s start thinking less about reports and more about actionable insights! Join this session to discuss.

12:30 PM - 2:00 PM
Lunch & Afternoon Keynote

Lessons in Driving Data Science and Analytics Transformation

Speaker: Chris Chapo, Gap

Description: While new companies have the luxury of building their organization with data at its core, for other legacy or larger global organizations, implementing a transformation around data science is a more herculean task. In this session, Chris Chapo will share his lessons and insights from driving a data science & analytics transformation across a global enterprise where understanding the customer is key to success. From how to focus on data quality and consistency, to driving new initiatives that deliver a customer benefit like personalization, Chapo will share his lessons for how to build an effective data program no matter where you are in your journey.

2:00 PM - 3:15 PM
Conversation #3 Topics

How to lead and inspire novices on good testing, without being a dream crusher

Moderator: Danielle Harvey, Wyndham

Description: Getting people excited about testing is nirvana for any testing lead, but it’s a delicate balance between creating enthusiasm and managing expectations. This conversation will center around the best way to educate and train testing novices to strategically scale your testing program.

All Things Mobile Testing

Moderator: Matt Hudson, Belk

Description: Mobile-specific A/B testing, multivariate testing; responsive vs. adaptive; geo-targeting; geo-fencing: this conversation will get into it all! If you’re testing on mobile either via mobile web or an app, join this session to discuss what’s working, what isn’t, and how to get the best results!

Testing Ideation: What to Do When the Well Runs Dry

Moderator: Scott Olivares, Cloudflare

Description: Picture this: your testing program is 2 years old; you’ve had some great wins–first the low-hanging fruit, then the less obvious testing ideas. Now, the furious flow of ideas is more like a trickle. Join this conversation to discuss how to come up with exciting new testing ideas, from conducting qualitative research to talking to your customer service team.

Navigating Testing in Regulation-Heavy Industries

Moderator: SunTrust

Description: Red tape weighing you down? Then you’ll want to join this conversation to discuss testing in industries that not only have to deal with cumbersome internal processes, but also federal regulations. Discussion topics include: streamlining the approval chain, code workarounds, getting to bolder tests and more.

Beyond Digital: Measuring Offline Experimentation

Moderator: Aaron Baker, Leading Media Company

Description: Consider the full customer journey: only part of that journey happens through digital channels, leaving a world of offline channels ripe for experimentation. But your traditional targeting and randomization techniques aren’t valid offline. Join this session to discuss the challenges of experimenting beyond digital, what solutions are being put forth, and ideas to effectively measure and experiment on these channels.

Customer Experience Starts with Empathy

Moderator: Scott Voigt, FullStory

Description: A great customer experience is no longer just “nice to have”, it will make or break your entire business. In this conversation we’ll discuss how empathy is the keystone to creating memorable customer experiences that impact results that matter: enthusiastically happy customers and exponential growth.

3:15 PM - 3:45 PM
Afternoon Networking Break

A short break between conversations to network with other attendees or swing by Brooks Bell experts or any sponsors to answer any questions you might have.

3:45 PM - 5:00 PM
Conversation #4 Topics

Be a Change Agent: How to preserve and prioritize your testing expertise during a re-org

Moderator: Kiki Burton, Credit Karma

Description: The dreaded re-org! Every business has them (or will!), so join this conversation to discuss how to preserve testing fundamentals in a changing structure. We’ll cover the risks, what has to be protected, and how can you bend and flex while staying true to testing principles.

UX-led Testing: How to use User Research and User Experience to develop your testing program

Moderator: Tim McKittrick, Enterprise Rent-a-Car

Description: In many organizations, there’s plenty of friction between the creative team that owns the brand, and the team that owns testing. This session will dive into the ways that Enterprise’s UX-led approach to testing has improved buy-in and validated the role of testing in improving the customer experience.

Bridging the Optimization Gap between Product and Marketing

Moderator: Manan Singh, LendingTree, Inc.

Description: Who owns optimization: product, marketing, both or neither? No matter who owns it, collaboration is paramount to ensuring the best user experience throughout the full conversion path. Join this conversation to discuss various business models, how to structure a program for experimentation that transforms the business, and how to ensure that no matter who owns optimization, the team isn’t losing site of the bigger business goals.

Testing ROI in a Lead-Generation Focused Company

Moderator: Jessica Xu, Oppenheimer Funds

Description: When your experimentation success depends on a conversion that can happen weeks or months down the road, how do you show the revenue impact? This conversation will dive into measurement and analytics, connecting testing to revenue impact, and using personalization to promote lead generation.

Server-Side Testing Vs. Client-Side Testing

Moderator: Andy Nelson, Chewy.com

Description: Companies are increasingly looking to server-side testing for security and performance reasons and to run experiments across their stack. While server-side testing opens up new avenues, it can also decrease speed and velocity in some settings. In this conversation, we’ll discuss the pros and cons of each method and what to consider when deciding whether client-side, server-side or a mix of both are right for your company.

Proactive Personalization: Data Driven 1:1 Experiences

Moderator: Jake Makler, Quantum Metric

Description: Personalization to date has been largely reactive: based on crunching historical data (ever see an ad for that t-shirt you just bought?). So how can you make personalization more proactive? How can you listen to signals from visitors and trigger personalized experiences? Join this conversation to discuss the data needed to proactively personalize, when to point customers to different channels, where it fits in your digital strategy, and even how to use it to predict behaviors and reduce churn.

5:45 PM - 6:15 PM
Transportation to Click Dinner

Leaves from The Umstead Hotel & Spa

6:15 PM - 10:30 PM
Click Dinner

The Dillon – 9th Floor Rooftop

9:00 AM - 10:15 AM
Breakfast & Keynote
10:15 AM - 11:30 AM
Conversation #5 Topics

Creating Institutional Memory: Preserving precious historical experimentation data to inspire future innovation

Moderators: Praveen Gujar, LinkedIn

Description: Whether win, loss or flat, there’s always something to learn from a test, and these learnings should be captured and cataloged. Join this conversation to discuss how to put a process in place to preserve these learnings and build upon them to improve your testing program.

Metrics that Matter: Discovering and Managing KPIs

Moderator: Tommy Guy, Microsoft

Description: Testing programs in an organization with diverse product groups present challenges for identifying and managing KPIs. This conversation will uncover how to identify KPIs that are sensitive and that correlate with ultimate business objectives. We will also explore challenges around metric definition standardization across larger organizations.

Testing for B2B

Moderator: Andrew Baldasare, Thermo Fisher Scientific

Description: B2B, B2C, now there’s even B2E…regardless, there’s a lot of nuance to testing with different target audiences. This conversation will dive into the challenges that B2B testers face, such as testing with less traffic and without a dedicated team; identifying personas and personalizing; using testing as a proof-of-concept and best practices for sharing results that get buy-in.

Brave New World: Globalization In Testing

Moderator: Brandon Brackett, Salesforce

Description: Testing on its own has its share of challenges–throw globalization into the mix and now you have language and culture barriers, time zones to contend with and massive hurdles to jump to scale your program. Join this conversation to discuss how your team and your peers are tackling knowledge-sharing, scaling and maintaining testing standards around the globe.

New Frontiers in Experimentation: How Optimizing your Product Delivers New Competitive Advantage

Moderator: Paul Colombo, Optimizely

Description: Companies like Uber, Square and Venmo are driving massive growth through the product experience. These companies have created a competitive advantage by studying the entire customer experience and quickly working to improve the product journey. But for many companies, simply defining a product remains challenging, and testing the concept of product is even more difficult. Join this conversation to discuss the transformation of testing from a marketing-focused activity to a product activity, the challenges that presents, and how to best ensure that the customer experience isn’t lost when optimizing your product.

11:30 AM - 11:45 AM
Networking Break

A short break between conversations to network with other attendees or swing by Brooks Bell experts or any sponsors to answer any questions you might have.

11:45 AM - 1:00 PM
Conversation #6 Topics

Changing the Product and Business mindset to embrace A/B testing: Challenges and Solutions

Moderator: Gurudev Karanth, Target

Description: It’s the same story every time: the product world wants quick answers from tests so they can move quickly, but calling a test too soon can lead to false positives or negatives. Join this conversation to discuss how to encourage proper testing to promote data-driven decision making in the product and business worlds.

You Have Executive Buy-In. Now what?

Moderator: Mary Jane Drobka, Slack

Description: As experimentation becomes more widely understood and accepted among the C-suite, one hurdle is taken care of. But what’s next? This session will dive into the issues of implementing a successful testing program, including being measured against metrics you don’t control, getting to yes with security teams, and using testing to give voice to under-resourced teams.

Building An Experimentation Program From The Ground Up

Moderator: Amy Smessaert, Change Healthcare

Description: Starting a program from scratch at a new company is a daunting task, fraught with resource constraints, budget limitations and an uphill battle to gain support for testing. Join this conversation to discuss how to prioritize and compromise to grow, and how to how to get scrappy to find success.

What’s on your mind – Testing Therapy

Moderator: David Rose, Gap / Claire Schmitt, Brooks Bell

Description: Now that you’ve got tons of great new ideas from your Click discussions, let’s get those remaining questions answered! This conversation will dive into those burning questions you still have so you can walk away with feeling good about your new program path and direction. Remember…you are not alone! Testers unite!

Unifying Disparate Customer Data to Enable 1-to-1 Personalization

Moderator: Matt Thompson, Evergage

Description: As marketing and customer engagement has evolved over the last 20 years, so has the complexity of the marketing technology landscape. Most enterprise businesses have a diverse martech stack with numerous systems holding valuable customer information, but most of this data is not unified nor actionable.

Join Matt Thompson, Sr. Director of Partnerships at Evergage, to discuss how to unify your siloed customer data, create 360-degree profiles for all of your customers, and act on those profiles in real time to drive cross-channel personalization.

1:00 PM - 2:00 PM
Lunch & Closing Keynote

The PICA Protocol™: Your Prescription for Healthy, Actionable Data Storytelling

Speaker: Lea Pica

Description: Are your data visualizations inspiring action or obscuring your valuable insights? If your stakeholders are falling asleep instead of singing your praises, you may not have the tools to deliver data stories in an actionable way. This session will empower you with a powerful, proprietary data storytelling methodology that informs decisions, sparks ideas, inspires action, and makes YOU indispensable.

2:00 PM - 4:00 PM
Brooks Bell Office Hours & Sponsor Demos

Grab a 30 minute slot to discuss testing program concerns or to see a product demo with Brooks Bell or one of the event sponsors.