A/B and multivariate tests are designed to let marketers know what messaging, content, offers and visual elements contribute to a lift in conversions. In effect, these testing techniques let customers signal what variations in messaging, images and other elements on a website or other digital channel led to a conversion, such as registering with a site or making a purchase.

What’s the greatest benefit from A/B and multivariate testing?  “An increased proclivity toward data-driven decision making,” says James Meyers, Gartner principal research analyst. “Marketers gain confidence in delivering customers’ preferred experiences when they have data that supports the decision.  After all, test results are numerical proof that customers prefer one experience over another.”

Surprisingly, many marketing organizations have yet to deploy technology to carry out these tests. Gartner’s Marketing Technology Survey of 206 marketers found that only 33% were regularly using A/B or multivariate testing technology and 28% were in the process of deploying a solution.

AB testing

 

The challenge? These tests  require a commitment of analytical and creative talent and a discipline to testing variations with content and design. “Expectations from A/B and multivariate tests are often unrealistic, inspired by spurious anecdotes (e.g., “A retailer changed the color of the ‘Buy Now’ button and sales doubled!” says Martin Kihn, research VP and author of the research report, Use A/B and Multivariate Testing to Improve Marketing Programs (subscription required.)

Marketers can avoid these pitfalls. Research Director Jane-Anne Mennella advises marketers who are focused on customer experience to use a blend of user experience, customer experience and marketing analytics, such as funnel analysis, page drop-off rates, to determine where to focus testing. (For best practices, Gartner clients can  see Jane-Anne’s report, Optimize Customer and User Experience Using A/B and Multivariate Testing.)

 

1 Comment
  1. February 26, 2018 at 12:58 pm
    Jyoti says:

    A/B testing or split testing can help to figure out the right way for marketers doing anything for their marketing strategies. Marketing professional can avoid any pitfall by implementing ab testing in their strategy. Thanks Anna for sharing this survey. Keep doing good work.

Leave a Comment

Your email address will not be published. Required fields are marked *