Event-driven decision making in commerce: a design for interpreting signals as they arrive

1Context and motivation

A commerce system changes state constantly: a price is edited, a product sells out, a card is declined, a carrier falls behind, a shopper rewrites a search query three times. Each of these is an event, and each carries information about what the next decision should be. When machine learning is run as a batch process, that information is folded into daily or weekly aggregates and reaches the storefront late.

Batch processing has three costs in this setting. The interval between a signal and the action it should trigger can exceed the life of the opportunity or the problem. Aggregation discards the ordering of events, which is part of what an analyst needs to separate cause from coincidence; claims about effects from observational data hold only under explicit assumptions, and timing is one of the things those assumptions rest on¹. And when checks on models run offline, a drifting model keeps acting until the next review.

We propose a design in which decisions are driven by events as they arrive, with the constraints that protect shoppers and merchants evaluated on the decision path. The design has two layers: an Event Signal Fabric (ESF) that senses and validates, and a Decision Orchestration Layer (DOL) that estimates, chooses and acts. This paper describes both as a worked design, from the literature, without measurements of our own.

2Design principles

• Every state change is an event. Events carry a schema version, an event time, an idempotency key, and consent and residency flags.
• Sensing is separate from acting. The ESF never writes to the storefront; the DOL never reads raw events. This keeps a fault in one layer from silently corrupting the other.
• Effects before scores. Where a decision asks what an action would change, the DOL uses effect estimates from experiments or quasi-experiments^{2, 3}.
• Constraints at decision time. Consent, exposure, latency and cost limits are checked before an action, with a safe default when a check fails.
• Human review for high-impact policies, with an explanation attached to each recommendation⁴.
• Full trace. Every action links back to the events, features, model and policy versions behind it⁵.

3Architecture

3.1Event Signal Fabric

Ingest. Producers publish to a partitioned, durable log. A schema registry rejects events that break their contract, and rejected events go to a dead-letter queue rather than being dropped.

Normalisation. Events are deduplicated by idempotency key and windowed by event time with watermarks, so that late arrivals are placed correctly rather than counted twice or lost. These are standard stream-processing techniques⁶. Each event is then enriched with region, device and customer-group attributes.

Signal store. Online and offline features share one definition, and each feature carries lineage and a freshness target set by its owner.

Quality monitors. Null rates, volume anomalies and distribution drift are tracked per critical field. Drift is the Kullback-Leibler divergence between a current window and a trailing baseline⁷; thresholds are configuration, not constants of the design.

3.2Decision Orchestration Layer

Causal services. Heterogeneous effects of promotions and placements are estimated with honest causal trees, which give valid intervals per subgroup². Where a region or category cannot be randomised, synthetic controls supply a comparison³.

Predictive models. Gradient-boosted trees serve tabular features⁸. Transformer-based sequence models⁹ serve ordered behavioural events where the latency budget of the decision allows; otherwise the tree model is used. Seasonal time-series models provide baselines for traffic and demand¹⁰.

Policy engine. Contextual bandits allocate exposure among content, promotions and layouts¹¹. Decisions that span several levers at once (price, assortment and delivery promise) go to a planner with linear or quadratic constraints on margin, inventory and capacity.

Safety layer. Consent and residency, eligibility rules (age, legal restrictions, regional policy), exposure floors and the remaining latency budget are checked before any action. On failure the layer returns a cached default.

Connectors. Merchandising, pricing, content, fulfilment orchestration and support tooling, each with idempotent writes and a rollback path.

4From signals to decisions

4.1Assortment

Repeated reformulations of a search and growth in wishlist additions for a category may indicate demand the catalogue does not yet meet. The DOL treats such a pattern as a hypothesis, not a fact: it is checked against a synthetic control built from comparable categories or regions before any assortment change is recommended³. External search-trend indices can sharpen short-term forecasts¹², but a model that relies on them alone can drift far from reality when search behaviour changes¹³, so they enter only as one feature among first-party signals.

4.2Pricing and promotion

Promotion effects are estimated per customer group² and allocated by a bandit. A promotion is not shown for items whose stock forecast suggests it would sell out before the promotion ends; this is our design rule, not a cited finding. The specificity of personalised offers is itself a decision. A field experiment on retargeting found that ads showing the exact product a person had viewed underperformed generic ads on average, and outperformed them only once the person's preferences had narrowed¹⁴. The design therefore caps product-specific personalisation for shoppers early in a search.

4.3Payments

Decline codes are tracked per issuer and payment method. A rise in declines for one method triggers an offer of another method at checkout for affected shoppers. Disputes, refunds and support contacts feed a trust signal for each shopper and merchant; we weight it because trust in the seller is a documented determinant of online purchase intention¹⁵.

4.4Delivery promises

Demand signals are joined with fulfilment constraints. When forecasts show congestion at a node or carrier, the planner shortens exposure for affected items or widens the promised window rather than making a promise the network is unlikely to keep.

5Stability and experimentation

Event-driven systems can overreact. A short spike in a noisy signal can move exposure sharply, and the next spike can move it back. The DOL damps this in two ways. A feedback regulator, after the adjustment schemes of process control¹⁰, limits how fast exposure can change, and dampening increases when observed volume leaves its forecast band. And a bandit's exploration is reduced for groups with sparse data.

A second risk is starving options of exposure. Patil and colleagues study a bandit in which every arm must receive at least a given share of pulls, and analyse the regret of algorithms that meet that constraint¹⁶. We apply the same idea to exposure across customer groups and regions.

Any claim that a policy helps must come from an experiment. Each experiment is registered with its hypothesis, primary and guardrail metrics, minimum detectable effect and rollback criteria before launch⁵. Pre-experiment covariates reduce variance (CUPED)¹⁷, and always-valid inference allows the continuous monitoring an event-driven system needs without inflating false positives¹⁸.

6Observability and governance

The system reports four kinds of health: decision health (latency, timeouts, fallback rate, exposure by group), data health (freshness, nulls, schema rejects, drift), model health (calibration, stability of effect estimates, out-of-distribution inputs) and business outcomes (revenue per session, contribution margin, cancellations, promise accuracy). Trace IDs run from ingest through features and decision to render, and each decision records the policy version, model hash and feature snapshot it used, so that any decision can be replayed.

Operators see SHAP attributions for tree-model recommendations⁴ and an alert when a model's top drivers shift. Overrides are logged and become labelled data. Reports to merchants give exposure rationale and effect estimates with confidence intervals; where aggregates are released outside the company, differential privacy bounds what they reveal about any one shopper¹⁹. Consent and residency are enforced at ingest and again at decision time; manipulative interface patterns are excluded by policy.

7Signal map

Signal family	Decision informed	Guard before acting	Source
Search reformulation, wishlists	Assortment changes	Synthetic control check	3
External search trends	Demand forecasts	Used only with first-party signals	12, 13
Clickstream, cart edits	Personalised offers	Specificity cap by stage	14
Promotion response	Promotion allocation	Effect intervals per group, exposure floors	2, 16
Traffic and demand series	Baselines and anomaly flags	Forecast band, damped response	10
Payment declines, disputes	Payment method offers, trust weighting	Consent and eligibility checks	15
Field distributions	Model suspension or fallback	Drift threshold set per field	7

Source: our design, with the works cited in the table.

8Evaluation protocol

We have not measured this design. To support a claim of effect it would need: calibration and sensitivity analysis offline, including placebo tests for every synthetic control³; pre-registered online experiments against a batch-driven baseline, analysed with always-valid inference and CUPED and with guardrails on latency, cancellations and exposure by group^{18, 17, 5}; and replay of recorded event streams through simulated outages and traffic spikes.

As a hypothetical scenario of the intended behaviour: a store whose main carrier began missing delivery windows in one region would see promise accuracy fall in the fulfilment signals; the planner would widen the promised window for that region or route orders to another carrier. Whether that reduces cancellations is a question for the experiment, not an outcome we report.

9Limitations

• No measurements. The paper reports no latencies, availability figures or effect sizes; it argues from structure and cited work.
• Confounding. Event ordering helps but does not remove unobserved confounders¹; natural experiments occur unevenly across markets.
• Sparse data. New markets and small categories give wide intervals, so the design falls back to conservative defaults there.
• Attribution. Several decisions often touch the same order (an offer, a layout, a promise). The design does not separate their contributions.
• Fairness definition. Exposure floors are one definition of fairness among several; they do not equalise effects across groups.
• Adversarial traffic. Bots and coordinated abuse distort behavioural signals; the design relies on detection it does not describe.
• Operator load. Review gates and explanations cost time, and how operators interpret attributions has not been studied for this system.
• Energy cost. Compute and energy use are not yet among the objectives the planner weighs.

10Conclusion

We have described a design in which commerce decisions follow events rather than batch schedules: a signal layer that validates and monitors what it receives, and a decision layer that estimates effects, allocates under exposure and latency constraints, damps its own reactions, and records enough to replay every choice. Each component addresses a known failure of batch-driven machine learning and rests on established methods. Whether the design improves outcomes is for the evaluation above to decide.