Continuous market sensing from commerce signals: separating demand from noise

1Context and motivation

Market research is often episodic: a quarterly survey, a panel, a commissioned report, a sales review. Between those points, intent shifts during promotions and shocks, demand moves between search, social and marketplace channels, and some demand never becomes visible in sales because stock, delivery or payment friction stops it. A commerce platform already observes many of these signals continuously, as a side effect of operating.

We describe a design that reads those signals continuously, across regions and categories, and passes confirmed changes in demand to the systems that act on them: assortment, promotion, localisation and delivery promises. The hard part is not detecting a change. A spike in searches can be a bot, a holiday, a reformulation caused by a broken filter, or a real shift in demand. A drop in sales can be falling demand or a stock-out. The design is built around confirming a change before acting on it. We describe the design and its reasoning; we have not measured its effects.

2Design principles

• Event first. State changes arrive as events with contracts; batch processing supplements them.
• Confirm before acting. An apparent change in demand is checked against confounders and against independent signals before any decision uses it¹.
• Effects over associations. Where an action is taken, its effect is estimated by cohort, not inferred from correlation².
• Several horizons. Nowcasts for immediate decisions, forecasts for capacity and assortment.
• Several resolutions. Signals are kept at product, category, corridor, region and macro levels.
• People confirm high-impact findings. Analysts review recommendations above a set impact level, with an explanation of the model's output³.
• Traceable. Lineage runs from raw signal to decision.

3Architecture

3.1Ingest and data contracts

Behavioural, operational and external signals arrive on streaming pipelines and are checked against versioned schemas for types, enumerations and null rates. Each event carries consent and residency flags, and processing stays in the event's region. Events can arrive twice or late, so the ingest layer deduplicates and uses event-time watermarks to decide when an aggregate is complete⁴. Each signal has a freshness target matched to its use: behavioural signals that drive same-day decisions need minutes, while macro indicators can be daily.

3.2Canonical signal store

Conformed entities (user, session, product, merchant, corridor, payment instrument) give every model the same definitions. A feature store serves the same values online and offline, with an owner, refresh cadence and quality monitors per feature. Distribution drift on key fields is measured as the Kullback-Leibler divergence from a reference window⁵.

3.3Causal and predictive services

• Forecasting and nowcasting. Seasonal time-series models for medium-horizon demand⁶. Gradient-boosted trees for fast tabular scoring⁷, and Transformer sequence models for high-frequency behavioural streams where latency allows⁸. Hierarchical models share strength across regions and categories where data is sparse.
• Synthetic controls. When a region or category shows an apparent change and no experiment is running, the expected path without the change is built from a weighted combination of comparable units, and the observed path is compared with it. Placebo tests, which apply the same procedure to units where nothing happened, show how large a gap arises by chance⁹.
• Per-cohort effects. When an action is taken in response to a signal, its effect is estimated per cohort with honest tree-based estimators².
• Causal graphs. Each recurring decision has a written causal graph naming its confounders, for example stock-outs that suppress sales of a product whose demand is rising¹.
• Anomaly detection on seasonally adjusted series flags sudden shifts for confirmation.
• Explanations. Tree models report local feature contributions, so an analyst can see which signals moved a score³.

3.4Decision layer and action connectors

Confirmed changes feed constrained decisions: a contextual bandit allocates promotional exposure, with minimum-exposure quotas so that small merchants and new products are not starved of the impressions their estimates need¹⁰; planners coordinate assortment, promotion and supply. Connectors deliver to merchandising tools, content management, promotion rules and supply orchestration, each with a rollback path to the previous state.

4Signals and what can mimic them

Table 1 lists the signal families, what a change in each may indicate, what else can produce the same change, and the check the design applies before acting.

Signal family	A change may indicate	Can also be caused by	Check before action
Search queries and reformulations	Emerging or unmet demand	Bots, broken filters, holiday terms	Bot filtering; seasonal adjustment; synthetic control on comparable categories9
Clickstream, dwell, cart edits	Stage of the buyer's decision	Page changes, latency	Compare with a holdout of unchanged pages11
Sales and conversion	Demand change	Stock-outs, payment declines, delivery fees	Causal graph with stock and payment as confounders1
Stock, substitutions, promise accuracy	Supply limits on visible demand	Carrier outages	Separate supply-side from demand-side series
Authorization rates, declines, disputes	Payment friction, loss of trust	Issuer policy changes	Per-issuer and per-instrument breakdown2
Exchange rates, holidays, local events, weather	External demand shifts	Calendar effects already in the baseline	Seasonal models with event regressors6
Search interest and social trend summaries	Brand or category momentum	Campaigns, coordinated posting	Require corroboration from a behavioural signal
Support topics, refund reasons	Product or localisation problems	A single incident or outage	Track by topic over time, not as single spikes

Source: our design, with the works cited in the table.

Two of these families need a note on interpretation. Search focus (how narrow and consistent a buyer's queries become) is used as a signal of where the buyer is in the decision, not only of what they want. A field experiment by an online travel firm found that ads showing the specific products a consumer had viewed were, on average, less effective than generic ads, and stopped underperforming only when browsing suggested the consumer's preferences had narrowed¹². The design therefore treats a shift towards specific content as appropriate only once signals indicate narrowed intent. Trust signals (disputes, refunds, support topics) are tracked because online purchase intention depends on trust in the seller, alongside the usefulness and ease of use of the site¹³.

5A worked model of multi-signal confirmation

The design acts on a suspected demand change only when it is corroborated by more than one independent signal family. Table 2 shows why, and at what cost, under stated assumptions.

Rule	Detectors	False alarm probability (computed)	Detection probability (computed)
Act on any single detector	1	0.05	0.80
Act when 2 of 2 agree	2	0.0025	0.64
Act when at least 2 of 3 agree	3	0.00725	0.896

Source: Worked model; inputs in the table.

For 2 of 2, the false alarm probability is 0.05 x 0.05 and the detection probability 0.80 x 0.80. For at least 2 of 3, the false alarm probability is 3 x 0.05^2 x 0.95 + 0.05^3 and the detection probability 3 x 0.80^2 x 0.20 + 0.80^3. Under these inputs, requiring two of three independent detectors both lowers false alarms and raises detection compared with a single detector, while requiring two of two buys fewer false alarms at the cost of missing more real changes. The result depends entirely on the independence assumption. A bot campaign that inflates both search volume and social mentions defeats it, which is why Table 1 pairs external signals with behavioural ones that bots are less likely to move together.

6From sensing to action

• Assortment. A rising cluster of reformulated queries and wishlist additions, confirmed against a synthetic control, becomes a recommendation to merchandising to source or promote matching products. The recommendation is suppressed for products with high stock-out risk.
• Promotion. Price response is estimated from experiments, not from observational sales that seasonality confounds. Promotional exposure is allocated by the constrained bandit, with guardrails against excessive discounting.
• Localisation. When reformulation patterns or support topics in one language diverge from others, the content system tests alternative copy for that locale.
• Delivery promises. When forecast congestion threatens promise accuracy for a corridor, exposure of affected products and their promised windows are adjusted.
• Payments. Changes in decline codes by instrument and region are separated from demand changes, so that a payment problem is not read as falling demand.

7Experimentation and evaluation

Before any outcome could be claimed for this design, each sensing-to-action path would need to pass the following protocol.

• Pre-registration of hypothesis, target metric, guardrails and minimum detectable effect in an experiment registry¹¹.
• Sequential testing with always-valid p-values, so that monitoring a running test does not inflate false positives¹⁴.
• Variance reduction using pre-experiment values of the metric as a covariate¹⁵.
• Offline evaluation of new policies with inverse-propensity-weighted estimates on logged decisions, and synthetic controls with placebo tests where experiments are not possible⁹.
• Causal diagnostics: overlap and balance checks, and sensitivity analysis for unobserved confounding¹.
• Detector evaluation. The quantities that Table 2 assumes (false alarm and detection rates per detector, and the correlation between detectors) would themselves be estimated on labelled historical periods.
• Holdouts kept after launch to monitor drift and policy effects.

Metrics follow from the decisions. Revenue per session, contribution margin (revenue less fulfilment, payment and return costs), promise accuracy (one minus the absolute gap between actual and promised delivery dates divided by the promised window), cancellation rate by promise-accuracy decile, and the fairness gap (largest minus smallest estimated effect across protected cohorts). Thresholds for these are left as parameters, to be set from measured baselines.

8Regional adaptation

Regions differ in which signals they have, which rules apply and what their calendars contain. Some have sparse payment data and rich search data, others the reverse. The design handles this with per-region feature sets, language-specific tokenisation for search and support text, regional holiday calendars as regressors, hierarchical priors that share strength across similar regions, and per-region processing of personal data. Mobile-heavy regions run lighter models within the same latency budget.

9Quantitative and qualitative evidence

Behavioural signals say what changed, not why. The design keeps three qualitative inputs alongside them: short surveys and intercepts that label sentiment against which inferred friction can be checked; regional reviewers who annotate anomalies that models may misread, such as local holidays or language shifts; and merchant feedback, which can flag false positives and local constraints and becomes labelled data. Operators can pause or adjust a policy when qualitative evidence contradicts a model, and each override is logged.

10Governance and privacy

Each request carries one identifier from ingest through feature, decision and action; model version, feature snapshot and cohort tags are logged, and a replay harness can re-run logged decisions. An experiment registry records hypotheses and guardrails¹¹. A data catalogue records owners and sensitivity classes. New signals and cross-border flows go through a data protection impact assessment. The design collects only the fields a decision uses, hashes identifiers, keeps personal data in its region, honours opt-outs, and avoids interface patterns that push buyers towards choices against their interest. Merchants see when and why their products are promoted or throttled.

11Limitations

• No measurements. We have not measured whether this design detects real demand changes earlier, or whether acting on them changes any outcome.
• Unobserved confounding. Synthetic controls and causal graphs reduce confounding but cannot exclude it, and natural experiments are uneven across regions¹.
• Independence in Table 2. The worked model assumes detectors are independent given the true state. Correlated failures, from bots or shared upstream outages, would make multi-signal rules much weaker than the table suggests.
• Signal meaning is untested. Query focus as a stage signal rests on one field experiment in one industry¹²; whether it carries over to other categories is an open question.
• Sparse markets. Hierarchical priors reduce noise in small regions but can also pull real local differences towards the average.
• Attribution. Multi-channel journeys blur which signal or action caused an outcome.
• Adversarial input. Bots and spam can pollute search and social signals; detection reduces this but does not remove it.
• Operator judgement. How analysts read explanations and when they override models varies, and affects outcomes.

12Conclusion

We have described a continuous market sensing design whose core is confirmation: signals are checked against what can mimic them, against synthetic controls and placebo tests, and against each other before any decision uses them. Table 1 lists the confounders for each signal family; Table 2 shows, with hypothetical inputs, how a two-of-three confirmation rule trades detection against false alarms and how much it depends on independence. The design's own performance is unmeasured. The next step is to estimate detector error rates and their correlation on labelled historical periods, then run the evaluation protocol on one category in one region.