Skip to main content

Insights

2024 State of Business Experimentation: Measure up with analytical leaders

By: Randy Cohen and Helen Erikson

Published: October 07, 2024 | Updated: October 10, 2024

Read time: 6 minutes

Topics

Future technology consulting

Product

Test & Learn®

Industry

Retail

Table of contents

  • Introduction
  • Common themes are popping up across industries
  • Investment trends are closely linked to experimentation
  • Business experimentation has many benefits
  • Conclusion

Introduction

Today, the need for testing remains as relevant as ever.

Consumer preferences change quickly, so it’s key to adapt. But not all decisions can be made with your gut alone. Nearly half — as in the case of retail, as high as 46% — of ideas do not break even or result in proving an initial hypothesis, according to Mastercard’s 2024 State of Business Experimentation (SOBE) survey.

Testing can lead to not only greater innovations and efficiencies in your business, but also to higher ROI for new initiatives and investments. In an annual survey conducted by Mastercard to understand current business experimentation trends, the 2024 SOBE report revealed the impact experimentation can have. 

How does it work? Completed by over 100 Mastercard Test & Learn® clients around the world, this survey data is first aggregated and anonymized before it is benchmarked and indexed. With this report, you can:

  • Learn about emerging business trends 
  • View trends dissected by business function
  • Understand testing practices
  • See how testing can create business value 
  • Gain competitive insights and learn how and what other leading companies are testing

This year, the survey was split into three industry-specific variations. Below are some highlights of the reports.

Common themes are popping up across industries

Common themes are popping up across industries.
 

1. Pressure comes from many angles. According to the SOBE 2024 survey results, respondents cite competitive pricing as a top concern. Brands continue to compete on price, among other factors, to capture their share of consumers spend as they balance between passing the rising costs onto the consumer or risk losing out. For others, survey results indicate that it’s the pressure that rising labor costs are placing on budgets. In 2024, top businesses have relied on testing for strategies to better manage finances.

2. Businesses always want to win customers, but over 80% of retailers say that most of their analyses are not run at the customer level. Customer analytics plays a crucial role in understanding how customers respond to initiatives and improving it is becoming a top business priority in all industries. 

3. The goal is always to get the customer to come back. According to the survey, the FI industry’s top four strategic priorities involve engaging, managing, targeting and optimizing the customer journey. Investments in consumer insights lead to a better understanding of the clientele and an experience that will get them to return. Top companies are using business experimentation platforms that allow them to address these priorities and understand consumer preferences.

4. Technology is gaining a bigger role. By implementing new tech — such as mobile shopping and payment solutions, AI and predictive analytics that more than two-thirds of retailers say they use — businesses can better personalize, automate and enhance the customer journey while also finding ways to be more efficient. Simply just adding any tech won’t guarantee a better experience, but insights from those who test suggest it can result in a better choice. 

5. The loyalty program space is competitive. Look at the restaurant industry: 73% of those surveyed share that they are responding to competitor programs by enhancing their own. Loyalty programs are supporting sales by tracking consumer-permissioned data, targeting offers and growing the customer base. Brands say they’re using business experimentation to test effectiveness of new loyalty strategies before a global rollout. 

Investment trends are closely linked to experimentation

From retail and restaurants to consumer packaged goods companies (CPG) and financial institutions, companies are heavily investing in testing capabilities.

  • Retail: For retailers, it’s pricing that is receiving the biggest investment focus. Experimenting with price has led to pricing optimization and personalization, a potential win-win for all parties. Retailers are also focusing on merchandising strategies to create efficiency and drive sales. For instance, 59% are investing in space allocation and 52% are allocating resources to seasonal merchandizing. 

All results are based on self-reported data by a subset of Test & Learn® clients in 2024

  • Restaurant: Similar to retail, restaurants are concentrating on pricing as a response to the upward pricing pressure, indicating that they plan for heavier investments in 2024 to account for rising costs and the need to stay competitive. A majority of responders are further investing in testable areas like menu design — for example, 77% are introducing new products and 73% are trialing limited-time or seasonal products. Consequently, to market these items, 86% of restaurants are investing in new product or category marketing. The ultimate goal is for the diner to leave with a full stomach and a healthy wallet.

All results are based on self-reported data by a subset of Test & Learn® clients in 2024

  • CPG: Merchandizing tests are top of mind in 2024 for CPGs. Companies expect to invest the most in areas that lend themselves to testing before a larger rollout: seasonal merchandizing (86%) and space allocation (71%). Unsurprisingly, pricing and technology rank highly as priority areas that may also see stronger investments.

All results are based on self-reported data by a subset of Test & Learn® clients in 2024

  • Financial institutions: FIs are focusing their investments on areas that can be tested and refined. They are particularly interested in personalized marketing, especially in the digital space, enhancing card-related features such as rewards programs, and adopting new technologies like AI and machine learning. These investments are all geared towards improving the customer journey and overall experience.

All results are based on self-reported data by a subset of Test & Learn® clients in 2024

Test & Learn®

Start testing new ways to engage consumers.

Learn more

Business experimentation has many benefits

The most advanced retailers conduct hundreds of experiments a year, scaling up their testing practices, driving business impact and realizing value. Mastercard’s Test & Learn® is used by top brands and empowers business decision makers to improve their initiatives and understand their overall incremental impact.

In nearly all industries, at least a third of clients realized $15M+ in value through Test & Learn® and the top third of respondents conduct more than 50 tests per year

There are numerous advantages to conducting tests. Test & Learn® provides faster and more accurate results, saving the average user over 25 business weeks per year. Most brands realize higher ROI with more testing, as seen with restaurants, where 68% of the time, they change rollout decisions based on the results of Test & Learn®.

92%

of retailers using Test & Learn® generate an average ROI of 10x+

15x

is the average ROI generated by restaurants that use Test & Learn®

>33%

of financial institutions using Test & Learn® generate an average ROI of 13x+

100%

of CPG companies using Test & Learn® generate an average ROI of 12x+

All results are based on self-reported data by a subset of Test & Learn® clients in 2024

Conclusion

Business experiments remain crucial to identifying which ideas will actually work. And the opportunity to refine decisions from testing comes naturally. 

No matter your industry — retail, restaurant or financial institution — contact us to discuss the 2024 State of Business Experimentation survey results and see how your organization's approach to testing and analytics measures up to leading industry peers.

Written by

Randy Cohen

Director, business development, Mastercard Services

Helen Erikson

Manager, business development, Mastercard Services