Incrementality testing vs Media Mix Modelling (MMM)

Incrementality testing

Incrementality testing goes beyond simple A/B testing by offering a variety of test designs to understand the true impact of marketing efforts. These tests can range from geo-lift experiments, where specific regions are targeted with increased advertising to measure the lift in sales, to time-series experiments that assess the impact of marketing efforts over time.

Use-case example of Incrementality test: A marketing team believed Pinterest was contributing more to their sales than reports suggested. To verify this, for 30 days they raised their Pinterest ad spend in a few select states that mirrored their broader target market. This approach aimed to simulate what an overall budget increase might achieve nationally.

As a result, the marketing team discovered that Pinterest’s contribution to sales was over twice as effective as indicated by traditional attribution models. This significant uplift was measured by comparing the sales performance in the test regions, where Pinterest ad spending was scaled, against control regions with standard ad spending.

Benefits:

• Direct measurement: Provides a clear picture of the causal impact of specific marketing actions.
• Flexibility: It can be applied to various marketing channels and strategies.

Challenges:

• Time-intensive: Requires considerable time to set up, run, and analyze tests.
• Limited scope: Focuses on specific campaigns or channels, making it challenging to gauge the overall effectiveness of the marketing mix.
• Resource-heavy: Needs significant resources and a dedicated team for test design and implementation.

Marketing Mix Modeling (MMM)

MMM provides a macro-level analysis of marketing effectiveness, incorporating external factors such as economic trends, competitive actions, and changes in user behaviour. This approach allows for allocating marketing budgets across channels and over time, optimizing for long-term sales.

Use-case example: A multinational beverage company uses MMM to determine the optimal allocation of its annual marketing budget. By analyzing data from past marketing campaigns, economic indicators and competitive activities, MMM helps identify which marketing channels (TV, digital, print, etc.) have historically provided the best return on investment. Additionally, it forecasts the potential impact of reallocating budgets towards emerging digital platforms, aiding in strategic planning for the upcoming year.

Benefits:

• Comprehensive analysis: Offers a macro view of the marketing mix, allowing for strategic budget allocation.
• Predictive insights: Helps forecast the impact of future marketing investments.

Challenges:

• Complexity: The advanced models can be difficult to understand without a data science background.
• Cost: Implementing MMM solutions can be expensive, with costs reaching up to $100,000 per year.
• Best for large advertisers: Most suitable for companies with significant digital ad spend (over $1 million per month).

Modern MMM: Innovations in machine learning have enabled more dynamic MMM solutions, allowing for real-time data analysis and more accurate predictive modelling. Companies like SegmentStream, Pecan, and Nexoya are at the forefront, offering solutions that employ MMM, predictive analytics and machine learning. For example, SegmentStream’s Marketing mix optimization tool provides specific recommendations on how to relocate the budget within selected campaigns. Learn more.

What to choose: Incrementality testing or Media Mix Modelling?

Incrementality testing offers immediate, actionable insights for specific campaigns or initiatives, making it ideal for tactical adjustments. In contrast, MMM provides a comprehensive view of the marketing mix’s effectiveness, suited for strategic planning and long-term budget optimization.

Yet, it’s not an either-or decision. Integrating both methods can enhance marketing strategies. Solutions from providers like SegmentStream, which offer both Incrementality Testing and Marketing Mix Modeling, enable marketers to benefit from immediate campaign insights and overarching strategic analysis. This combined approach supports both immediate tactical moves and informed long-term budgeting, offering a comprehensive toolkit for marketing optimization.