What is incrementality testing?
Short answer: Incrementality testing is a controlled experiment that measures the true causal lift from your advertising — the conversions that happened because of the ads and would not have happened otherwise. You compare a group exposed to ads (test) against a comparable group that is not (control), and the difference is the real lift.
Key takeaways
- Incrementality testing measures the lift your ads actually caused — not sales that would have happened anyway.
- It works by comparing a test group (sees ads) against a control group (does not).
- Incremental ROAS (iROAS) is almost always lower than platform-reported ROAS.
- It is the gold standard for proving cause and effect, and the tie-breaker between attribution and MMM.
Every ad platform happily reports the conversions it “drove.” The problem: some of those buyers were going to convert no matter what. Attribution can over-credit ads because it only looks at people who already saw them. Incrementality testing fixes this by asking a sharper question: how many of these conversions would not have happened without the ad?
Why reported conversions overstate impact
Imagine you run a brand-search campaign and it reports 1,000 conversions. If 700 of those customers would have found you and bought anyway, the ad’s real contribution is only 300. Reported conversion tracking shows 1,000; incrementality shows the 300 that count. That gap is why teams that switch to incremental measurement often cut wasted spend.
How incrementality testing works
- Holdout tests: randomly withhold ads from a slice of your audience and compare their conversion rate to the exposed group.
- Geo experiments: turn ads on in some regions and off in matched regions, then compare sales. Popular because it needs no user-level data.
- Ghost ads / PSA tests: the control group is served a placeholder so the only difference is your ad.
The lift is simply: (test conversions − control conversions) ÷ control conversions, scaled to your audience.
Incremental ROAS (iROAS)
Once you know the true lift, you can compute incremental ROAS = incremental revenue ÷ ad spend. It is almost always lower than the ROAS your platform reports, because it removes conversions that would have happened anyway. A campaign with a glossy 6× reported ROAS can have a 1.5× iROAS — still profitable, but a very different story for budgeting.
Incrementality vs attribution vs MMM
- Attribution credits the touchpoints a converter saw — fast and granular, but can over-credit.
- MMM estimates channel contribution from aggregate history — privacy-safe and strategic.
- Incrementality proves causation with a control group — the most rigorous, and the tie-breaker when the other two disagree.
Limitations to know
- You need enough volume and a clean control group for the result to be statistically valid.
- Tests take time — typically two to six weeks — so they are not instant.
- Withholding ads from part of your audience has a short-term opportunity cost.
FAQ
What is incrementality testing?
A controlled experiment that measures the true causal lift from a campaign — the conversions that happened because of the ads and would not have happened otherwise — by comparing a test group against a control group.
What is incremental ROAS?
Incremental ROAS (iROAS) is revenue from the extra, ad-driven conversions divided by ad spend. It is usually lower than platform-reported ROAS.
How is incrementality different from attribution?
Attribution assigns credit to touchpoints a converter saw; incrementality uses a control group to prove how many conversions the ads truly caused.