Incrementality: Holdouts, Lift Math, and Decision Use

Updated September 20, 2026· 7 min read

Incrementality is the extra outcome caused by marketing compared with a valid counterfactual. Attribution assigns credit inside recorded paths. A pre-post chart assigns the whole calendar to the campaign. Neither one is a holdout.

Read this next to marketing attribution and conversion lift. Reporting lives in reporting and analysis.

Hypothetical equal-size holdout showing the lift and incremental-CPA calculations.

Designs and contamination

Design	How the control is formed	Best fit	Main failure
Randomized user holdout	Eligible people assigned to treatment or control	Platforms that can suppress delivery for the control	Identity gaps, households, noncompliance, and cross-device leakage
Geo experiment	Comparable regions show or withhold	Broad media or offline conversions	Travel between regions contaminates groups
Matched market	Markets chosen to look alike	When user randomization is unavailable	Markets still differ on unmeasured factors
Time-based pause	On versus off across dates	Cheap directional read	Seasonality and other channel changes

Google Conversion Lift is one incrementality product (not in every account). User-based studies split people. Geo studies split Google Marketing Areas and model contamination because someone can see an ad in a treatment region and convert in a control region. That leakage shrinks estimated lift.

Before reading any number, check:

• Eligibility. Only units that could have been treated belong in the test.
• Assignment and analysis. The control is chosen by the design, not by people who happened not to click. Analyze randomized groups as assigned so selective exposure does not replace randomization.
• Pre-period balance. Outcomes and mix should look similar before launch.
• Power. If the smallest useful effect is smaller than the test can detect, an inconclusive result is likely by design.
• Contamination. Shared codes, frequency caps that still leak, and travel between geos all move exposure into the control.

Lift math

When group sizes differ, use rates.

• Absolute CR lift = treatment conversion rate − control conversion rate
• Relative lift = absolute CR lift / control conversion rate
• Incremental conversions = (treatment CR − control CR) × treatment users Or, when groups are equal or already scaled: treatment conversions − control conversions
• iCPA = incremental treatment spend / incremental conversions
• iROAS = incremental conversion value / incremental treatment spend

Incremental treatment spend is the treatment group’s spend minus the comparable spend that would have occurred in control. In a pure ad holdout with no control spend, it is simply the treatment spend. If both groups receive different media levels, use the spend difference on the same scaled basis as the outcome difference.

If incremental conversions are zero, near zero, or the interval includes zero, do not divide spend by a tiny estimate and publish an iCPA. Report the interval and an inconclusive decision.

Attributed conversions are not this math. Last-click totals can rise while incremental conversions are zero.

Worked hypothetical holdout

Hypothetical equal-size randomized groups, analyzed as assigned. The control receives no campaign spend.

Group	Eligible people	Conversions	Conversion rate	Ad spend
Treatment	10,000	350	3.50%	$5,250
Control	10,000	280	2.80%	$0

• Absolute CR lift = 3.50% − 2.80% = 0.70 percentage points
• Relative lift = 0.70 / 2.80 = 25%
• Incremental conversions = 350 − 280 = 70 (also 0.007 × 10,000)
• Incremental spend = $5,250 − $0 = $5,250
• iCPA = $5,250 / 70 = $75

Those 70 extra conversions are the lift estimate for this window. They do not prove a particular ad, audience, or publisher caused them unless those factors were separately randomized and powered.

Write the spend decision for three outcomes before the study: positive lift, no lift, and an inconclusive interval.

Practical check

Write the causal question first. Protect the control. Check pre-period balance. Report the interval with the point estimate. Do not scale budget from attributed conversions alone.

FAQs

What is incrementality in marketing?

Incrementality is the causal difference between an assigned treatment group and a valid counterfactual, usually a randomized holdout or a comparable geo design. It answers whether extra outcomes occurred because of the activity, not which recorded click received credit.

How do you calculate conversion lift?

Absolute conversion-rate lift is treatment CR minus control CR. Relative lift is that gap divided by control CR. Incremental conversions are the rate gap times treatment units when groups are comparable. iCPA is incremental treatment spend divided by incremental conversions.

When is a lift result not causal?

A result is not causal when the design cannot support a counterfactual: for example, assignment failed, ads leaked into a randomized holdout, a matched-market design violated its identifying assumptions, or a pre-post or attribution delta was labeled as lift. An interval that includes zero is inconclusive.

How does incrementality differ from attribution?

Attribution assigns credit inside observed paths using a rule or model. Incrementality compares all outcomes in treatment versus control, including conversions with no recorded click. Use both: attribution for path reporting, incrementality for go or no-go spend decisions.