Cross-channel attribution with learned credit models, calibrated by experiments

1Context and motivation

A shopper may see a social post, click a search ad a day later, open an email, and finally type the store's address and buy. Attribution is the task of dividing the credit for that purchase among the touches that preceded it¹. The division matters because budgets follow it: a channel that receives little credit tends to lose spend.

The most common answer is still a rule. Last-touch attribution gives all of the credit to the final touch regardless of what came before², and first-touch, linear and time-decay rules are equally fixed in advance. Such rules do not represent interactions between channels, dependence on the order and timing of touches, or differences between users³. The research response has been data-driven attribution, which fits a model to observed journeys and derives credit from it.

There is a second, less comfortable line of research. Field experiments on advertising show that measuring its returns is hard even with randomisation⁴, and that observational methods often fail to recover the effects that randomised experiments measure⁵. A data-driven attribution model is an observational method. It can describe journeys well and still assign credit that does not match what would happen if a channel's spend changed.

This paper describes a design that uses learned credit models for what they do well, describing and decomposing journeys, and relies on experiments for what they cannot do, estimating incremental effect. We report no measurements of our own.

2Background

2.1Rule-based and data-driven credit

Data-driven multi-touch attribution was proposed to replace the single-touch rule with credit based on the contribution of each touch to the final decision, with stability of the estimate as an explicit goal alongside accuracy¹. An empirical model of channel consideration, visits and purchases that estimates carryover and spillover between channels gives channel contributions that differ significantly from those of the metrics in common use; a field study in the same work validated its estimate of a channel's incremental impact⁶.

2.2Cooperative game theory

The Shapley value treats channels as players in a cooperative game. Given a value v(S) for every coalition S of channels, it assigns each channel its marginal contribution averaged over all orders in which the coalition could be assembled⁷. It is the unique allocation with a set of fairness properties: the shares add up to the value of the full coalition, symmetric channels receive equal shares, and a channel that adds nothing receives nothing. The same construction underlies SHAP explanations of individual model predictions⁸.

The Shapley value says how to divide v; it does not say where v comes from. If v(S) is the observed conversion rate of journeys that touched exactly the channels in S, the resulting credit inherits every bias in who was exposed to which channel. Attribution can be framed as a causal estimation problem, with approximations drawn from cooperative game theory for cases where full causal estimation is not possible; when the causal assumptions fail, those approximations are better read as measures of variable importance than as effects⁹.

2.3Graphs and sequences

Graph-based attribution represents customer journeys as transitions between channel states and derives credit from that graph¹⁰. A common way to do so is the removal effect: the relative drop in the probability of reaching conversion when a channel's state is removed from the graph. Sequence models go further: recurrent and attention-based networks learn credit from the conversion prediction objective, using the order of touches and, in some designs, both clicks and impressions², or user context as a control for the estimated channel effects³.

2.4Experiments

Randomised experiments remain the reference for incremental effect. They are expensive for advertising because individual sales are volatile relative to the cost of an ad, so informative experiments can need very large samples⁴. Designs such as ghost ads reduce that cost by identifying, within the control group, the users who would have been exposed, which improves precision at a fraction of the cost of public-service-announcement controls¹¹. The general framework of controlled experiments, with pre-registered metrics and guardrails, applies to marketing tests as to product tests¹².

3Design

The design has five layers.

3.1Identity and consent

Journeys are assembled only from touches that can be linked to the same person deterministically, through a login or a confirmed email address, and only where the person has consented to measurement. We do not use device fingerprinting or IP clustering to link devices. This choice makes journeys shorter and less complete than probabilistic linking would, and the design treats that incompleteness as a known bias to be reported, not filled in by inference.

3.2Journey assembly

Each journey is an ordered list of touches with channel, timestamp, touch type (impression, click, visit, email open) and whether it ended in conversion within a fixed window. Non-converting journeys are kept: every credit model below needs them.

3.3Credit models, side by side

Four credit models run on the same journeys:

• Rule baselines. Last touch and linear, kept as reference points that any reader can reproduce.
• Shapley credit. Coalition values are estimated from journeys and divided by the Shapley value⁷.
• Graph credit. A first-order Markov graph of channel transitions, with credit from normalised removal effects¹⁰.
• Sequence credit. An attention-based sequence model trained to predict conversion, with credit read from its attention weights or from SHAP values over touches^{2, 8}.

Running the models side by side is deliberate. Where they agree, the credit is robust to modelling choice; where they disagree, the disagreement is reported, because it shows that the data do not determine the answer.

3.4Calibration against experiments

No credit model is allowed to change a budget on its own. For each channel, a randomised incrementality experiment (a geographic holdout, or a ghost-ad design where the ad platform supports one¹¹) estimates the channel's incremental conversions over a period. Each credit model's output for the same channel and period is compared with that estimate, and the ratio is stored as a calibration factor. Budget proposals use calibrated credit, and a channel with no recent experiment is flagged as uncalibrated.

3.5Budget proposals

The budget layer proposes reallocations from calibrated credit and the marginal cost of each channel. Proposals are bounded in size per period and each large shift is itself run as an experiment where possible¹². We do not propose an automatic reinforcement-learning allocator: with the measurement noise documented for advertising⁴, an allocator learning from attributed credit would chase noise and its own biases.

Table 1 summarises the components and the evidence each rests on.

Component	Problem it addresses	Evidence
Rule baselines kept as reference	Rules are arbitrary but reproducible	2, 3
Shapley credit over channel coalitions	Credit should sum to the whole and treat symmetric channels equally	7
Graph credit by removal effect	Journeys are sequences of transitions, not sets	10
Sequence models with attention	Order, timing and interaction between touches	2, 3
Carryover and spillover between channels	One channel's touch changes the response to another's	6
Calibration against randomised experiments	Observational credit can differ from incremental effect	9, 5
Ghost-ad or geographic holdouts	Advertising experiments are costly and noisy	4, 11
SHAP over touches for explanations	Operators cannot see why a journey's credit fell as it did	8

Source: Each row cites the work it rests on; the components are our design.

4A worked Shapley example

Table 2 gives coalition values for three channels: social (S), search (Q) and email (E). The values are inputs we chose for illustration, expressed as conversions per 1,000 journeys above a no-touch baseline. They are not measurements.

Coalition	v (input)
none	0
S	10
Q	20
E	5
S + Q	35
S + E	18
Q + E	26
S + Q + E	40

Source: Worked model; inputs in the table.

For three players, a channel's Shapley value is one third of its value alone, plus one sixth of its marginal contribution to each one-channel coalition, plus one third of its marginal contribution to the two-channel coalition of the others. For social, that is 10/3 + (35 minus 20)/6 + (18 minus 5)/6 + (40 minus 26)/3 = 12.67. Table 3 gives the results.

Channel	Value alone, v(i)	Shapley value	Shapley share of 40	Removal effect, v(SQE) minus v(without i)
Social (S)	10	12.67	31.7%	14
Search (Q)	20	21.67	54.2%	22
Email (E)	5	5.67	14.2%	5
Sum	35	40.00	all of v(SQE); shares rounded	41

Source: Worked model; computed from the inputs in Table 2.

Three points follow from the arithmetic. First, the values alone sum to 35, not 40: the channels interact, and the extra 5 is shared by the Shapley value according to each channel's marginal contributions. Second, raw removal effects sum to 41, more than the total, so they have to be normalised before they can be read as shares. Third, and most important, nothing in the computation checks the inputs. If the coalition values were read from observed journeys in which people already inclined to buy were more likely to see search ads, search would receive credit for that inclination^{9, 5}. This is why the design calibrates against experiments.

5Evaluation

Before any claim about this design's accuracy or its effect on return could be made, a deployment would need the following.

• Holdout experiments per channel. At least one randomised incrementality estimate per major channel per quarter, with the method recorded^{11, 12}.
• Calibration error. For each credit model, the gap between its credit and the experimental estimate, reported per channel. A model is preferred only if its calibration error is lower across channels and periods, not in one test.
• Stability. Credit for the same period recomputed on bootstrap samples of journeys; unstable credit is reported with its interval¹.
• Model disagreement. The spread of credit across the four models, reported alongside any single model's output.
• Identity coverage. The share of conversions with a linked journey, since unlinked conversions bias credit towards the touches that happen on the purchasing device.
• Budget outcomes. A reallocation proposed by the design is tested against the existing allocation in a geographic experiment before it is adopted.

We set no target values for these metrics; they depend on baselines we do not have.

6Limitations

• No measurements. We have run no experiments and fitted no models for this paper. Accuracy, return and acquisition-cost effects are all unmeasured.
• Credit is not effect. Shapley, graph and sequence credit are descriptions of observed journeys. Calibration narrows the gap to incremental effect only for channels and periods with recent experiments.
• Experiments are noisy and costly. Advertising experiments can need very large samples⁴. Small stores may never have enough traffic for informative holdouts, and the design then has nothing to calibrate against.
• Deterministic identity only. Consented, deterministic linking misses journeys that cross devices without a login. Credit for upper-funnel channels on other devices is likely understated.
• Coalition values grow exponentially. Exact Shapley values need a value for every subset of channels; with many channels this needs sampling or grouping, and sparse coalitions are poorly estimated.
• Attention is not explanation. Reading credit from attention weights in a sequence model is a modelling choice whose faithfulness is not established here.
• Platform constraints. Ghost-ad designs depend on support from the ad platform; geographic holdouts depend on regions that are comparable.

7Conclusion

We have described an attribution design in which learned credit models describe journeys and randomised experiments decide what that credit is worth. Shapley, graph and sequence models run side by side on consented, deterministic journeys; their disagreement is reported; and only experiment-calibrated credit can propose a budget change. The worked example shows both why the Shapley value is a principled way to divide credit and why it cannot fix biased inputs. The next step is to run holdout experiments on two channels and report the calibration error of each credit model against them.