Why Can a Supply-Chain Backtest Work and the Live Signal Still Fail?
September 14, 2026
Altsets
Research by Altsets Research
The live system receives relationships only after discovery, mapping, processing, and delivery, so reconstructed history can outperform production when backfills, observation delays, security changes, or offline-versus-online feature drift are ignored.
Data used:Altsets Supply Chain Intelligence: 90k+ entities, 400k+ relationships, 20+ years of history.
Key findings
- A modern historical database can be cleaner than the information stream available in real time because later relationship discoveries, revisions, and entity-resolution improvements can backfill or correct old records.
- SK Hynix's 2026 Nasdaq ADR illustrates why live production must preserve one economic company while adding and mapping new tradable securities without duplicating network exposure.
A supply-chain strategy can survive years of historical testing and still behave differently the moment it runs live. Some of that gap comes from ordinary execution. Alternative data adds another layer because the live system only receives relationships after they are discovered, normalized, mapped to securities, and delivered. A historical database viewed today can be cleaner and more complete than the information stream the strategy will actually receive tomorrow. The quant needs to test not only whether the feature predicted returns, but whether the production system can recreate the historical information conditions assumed by the backtest.
Backfilled knowledge can make history look cleaner than live reality
Suppose a relationship is eventually confirmed and inserted into several historical monthly snapshots because the economic relationship existed then. That reconstruction can be useful for business analysis, but a live strategy in those old months may not have known the relationship yet. If the backtest uses the reconstructed history without respecting observation time, the model gets a cleaner graph than the real strategy would have had.
The same issue can appear when a metric is revised or an entity mapping improves. A current database can know that two aliases refer to the same company, while the historical production pipeline may have treated them separately until the mapping was resolved. Recent point-in-time research warns about exactly this broader problem: historical databases can leak future knowledge when later revisions overwrite the record or when facts are indexed by the period they describe rather than the time they became usable. The live system cannot benefit from a correction before the correction exists.
Security mapping can change even when the economic company does not
SK Hynix provides a current example of why the live production layer needs entity and security awareness. The company began Nasdaq ADR trading in July 2026 while continuing to exist as the same underlying Korean operating company. A historical model tied only to one security identifier can miss the new tradable instrument when it appears, while a model tied too loosely to the company can accidentally duplicate exposures across the home share and ADR.
Live deployment needs an explicit rule for such changes. The company-level supply-chain features can persist across the security event, but execution, trading hours, currency, liquidity, and price history remain security-specific. The same problem appears with acquisitions, ticker changes, delistings, and spin-offs. A backtest can pre-clean those corporate actions perfectly after the fact, while a live system has to react to them as they happen.
Production latency should be compared with the signal half-life
A relationship feature that changes monthly can tolerate a slower ingestion path than a customer-event strategy whose signal decays in hours. The live pipeline should therefore measure when the source became public, when the relationship or event entered the system, when the feature was recomputed, and when the trading process received it. If the median total delay is longer than the measured signal half-life, the backtest is testing information the production system cannot monetize in time.
This is one reason supply-chain strategies should separate structural features from event features. The graph can be refreshed on a slower cadence while earnings releases, product announcements, and other fast catalysts are streamed against the latest approved graph. Rebuilding complex centrality or peer-group features on every headline can add latency without changing the underlying relationships. Production design should match the temporal behavior found in the research.
Live shadow features can detect research-serving drift
Before risking capital, the researcher can run the production feature pipeline in shadow mode and compare every live feature with what the research pipeline would have generated from the same information. Differences should be explainable by timing rather than silent implementation drift. A customer concentration score that differs because the live system is using a different edge filter is a bug. A score that differs because a new relationship became observable after the historical cutoff is expected.
Logging those differences creates a useful audit trail. It also allows the quant to determine whether live degradation comes from the economic signal or from the feature pipeline. Without that separation, a strategy can appear to lose alpha when the real problem is that the live graph is missing relationships, using stale mappings, or receiving data later than the backtest assumed.
The conclusion is that the live test begins with information parity
A backtest answers what would have happened under its historical information and execution assumptions. A production strategy only deserves to inherit that evidence when it can reproduce those assumptions closely enough in real time. For supply-chain data, the largest live gap can come before the order is sent: observation delays, backfills, entity-resolution changes, security mapping, and different offline versus online feature definitions can all make the live model consume a different graph from the one the backtest tested. Monitoring information parity is therefore part of monitoring the strategy.
The point-in-time feature-store guide explains how to preserve historical and live feature semantics. The signal-half-life guide explains why production latency should be judged against the speed at which relationship information decays.
For relationship definitions and evidence limits, read the Altsets methodology.