A campaign can show impressive-looking attributed conversions and still have contributed essentially nothing to actual incremental sales — if the customers it “converted” would have purchased anyway, through another channel or organically, the campaign’s attributed results are measuring correlation, not causation. Incrementality testing is the deliberate methodology for answering the harder, more important question: did this spending actually cause additional sales that wouldn’t have happened otherwise?
Why Attribution Data Alone Can't Answer the Causation Question
Standard attribution assigns credit to whatever touchpoint a conversion tracking system observed in a customer’s path, but it can’t distinguish between a touchpoint that genuinely caused the purchase and one that simply happened to occur before a purchase the customer would have made anyway — a customer who already intended to buy your product and searched your brand name directly would show up as a “conversion” attributed to that search, even though the search itself likely didn’t cause anything.
The Core Incrementality Testing Method: Holdout Groups
The most rigorous approach withholds a specific marketing activity (an ad campaign, a channel) from a randomly selected control group while the rest of the audience receives it normally, then compares actual business outcomes between the two groups — the genuine difference between the exposed and holdout groups represents the true incremental impact, isolated from customers who would have converted regardless of the specific marketing activity being tested.
Geo-Based Incrementality Testing
For channels or campaigns that can’t easily randomize at the individual level, geographic holdout testing — running a campaign in some markets while withholding it from comparable control markets — provides a practical alternative, comparing aggregate outcome differences between exposed and unexposed geographic regions rather than requiring individual-level randomization.
Platform-Provided Incrementality Testing Tools
Several major advertising platforms now offer built-in incrementality testing features, running holdout experiments directly within their own ad systems — these provide a genuinely more accessible entry point than building custom incrementality tests from scratch, though it’s worth understanding that a platform testing its own incrementality carries some inherent conflict of interest worth factoring into how much weight to give the results.
Why Incrementality Testing Matters Most for Brand and Retargeting Campaigns
Brand search campaigns and retargeting in particular are prone to capturing attribution credit for conversions that would have happened anyway, since both target audiences already showing strong existing purchase intent — incrementality testing on these specific campaign types often reveals a meaningfully smaller genuine incremental contribution than their attributed conversion numbers alone would suggest.
Building a Practical Incrementality Testing Program
- Start with your highest-spend, most attribution-suspect channels — typically brand search and retargeting — where the gap between attributed and genuinely incremental results tends to be largest and most consequential for budget decisions.
- Run tests for a sufficient duration to capture genuine business cycles and avoid short-term noise producing misleading conclusions.
- Repeat testing periodically rather than treating a single test as permanently conclusive, since incrementality can shift as market conditions, competition, and customer behavior evolve over time.
Using Incrementality Findings to Reallocate Budget
Channels showing genuinely strong incremental lift deserve continued or increased investment with confidence; channels showing attributed success but weak actual incrementality warrant either reduced investment or a genuine reconsideration of their role (perhaps serving a different purpose, like customer retention, rather than net-new acquisition) — incrementality data should directly inform these reallocation decisions, not simply confirm existing attribution-based assumptions.
Balancing Rigor With Practical Resource Constraints
Full incrementality testing requires genuine setup effort and some sacrifice of short-term optimization (deliberately withholding spend from a control group), which needs to be weighed against the value of the resulting insight — reserving rigorous incrementality testing for your highest-spend channels, where the insight’s value clearly justifies the testing investment, rather than attempting it uniformly across every minor campaign.
Where This Fits the Broader Strategy
Incrementality testing answers the causation question attribution data structurally can’t, revealing which marketing spend genuinely drives additional sales versus which merely captures credit for purchases that would have happened anyway. For the complete strategic framework, see our complete guide to data-driven marketing analytics.
A campaign’s attributed conversions and its actual incremental impact can be dramatically different numbers — incrementality testing is the only methodology that reveals whether spending genuinely caused the sale, not merely correlated with a purchase that would have happened anyway.
