Past-State Contamination¶

Version: v4 · 2026-08-30 · History
Prime #: 1411
Origin domain: Epistemology
Subdomain: temporal validity → Epistemology
Aliases: Retrospective State Contamination, As of Contamination, Future Information Contamination
Related primes: Temporal Dynamics

Core Idea¶

Past-state contamination occurs when information that became available only after a target time enters an artifact offered as valid for that time. The artifact may be a reconstruction, an estimate, a fitted model, a decision record, or an evaluation; what makes it a candidate is not its form but its claim, which is that it shows what was, or could have been, held at t. ^[1]

Five roles must co-hold. There is a target time t to which the artifact is indexed. There is the information set that had actually come into existence and reach by t. There is at least one item whose availability begins strictly after t. There is the assertion of as-of validity, which is what converts a mere late arrival into a violation. And there is a channel — a mutable database field, a join against a current table, a shuffled sample, a reviewer who has already read the ending — through which the late item reaches the artifact without leaving a mark on it. ^[1]

What the prime grades is provenance in time, not correctness. A contaminated artifact may be more accurate than an uncontaminated one, and often is, precisely because the extra information is real. Detection is therefore counterfactual rather than inspectional: one asks whether the artifact would have come out the same had the post-t items been withheld, and any divergence is the contamination itself. Nothing on the surface reveals it, because the surface is exactly what the artifact would look like had it been honestly built. ^[1]

Structural Signature¶

An artifact indexed to a target time and asserted valid as of it → an item whose availability begins after that index → a backward channel carrying the item in with no provenance boundary preserved → counterfactual divergence from the reconstruction the target-time information set could support. ^[1]

Recurring features:

Two independent clocks on every fact: the time the fact is about, and the time the fact became reachable. The pattern is definable only where the second clock exists and detectable only where it was recorded.
An as-of claim carried by the artifact — explicit in a backtest or a chart abstraction, merely implied in a narrative memo, but always the thing that makes lateness matter.
At least one input whose availability postdates the index, where availability rather than the period label printed on the input is what counts. A restated figure wears an old label and a new arrival time.
A channel with no boundary marker: overwrite-in-place storage, a random partition, a lookup against the live version of a table, an unblinded human reader. The crossing leaves no residue in the output.
Counterfactual bite. Where withholding the late items changes nothing, the discipline lapsed but the artifact did not.
Asymmetric visibility: the contaminated artifact beats its honest twin in reconstruction and matches it in prospect, so the defect announces itself only where those two regimes can be compared.
A mechanical repair — freeze, blind, or split by time — that closes the channel without anyone having to adjudicate the content of what crossed. ^[1]

What It Is Not¶

The prime is not an accusation of dishonesty. Almost every real instance is produced by convenience rather than intent: the current table was the one already loaded, the reviewer was whoever had the file, the split was the library default. Nor does it forbid the use of later evidence. A history written today may draw on everything the archive now holds, a post-mortem may name a cause nobody could see at the time, and a revised estimate may absorb a decade of subsequent data — none of that is contamination, because none of those artifacts claims its content was available earlier. The violation lives in the pairing of a late input with an as-of assertion, and it can be cured from either side: withdraw the input, or withdraw the claim. ^[2]

Several further things fall outside what the prime asserts. It does not grade severity; a decisive leak and a trivial one are the same kind of thing, and the abstraction is silent on how much either mattered. It is not confined to quantitative work, since a witness statement, a clinical note, a design rationale, and a court's reasonableness finding are all contaminable and none of them contains a number. It does not require that the artifact was ever used, the defect being a property of how the thing was built rather than of what followed from it. It does not imply the artifact is wrong about the past; an account can be scrupulously time-respecting and simply mistaken, which is a different failure with a different repair. And it supplies no threshold for when information counts as available — whether a document filed but unread, or a figure computable but never computed, was within reach at t is a question the domain has to settle for itself. ^[3]

Broad Use¶

Quantitative finance. A strategy is tested against a fundamentals history whose figures have since been restated, an index whose membership was announced days after the date it is attributed to, and a universe from which delisted names have quietly been dropped.

Machine learning and forecasting. Scalers, imputers, and feature selectors fitted over a whole dataset before it is split; random rather than chronological partitions in time-series work; label windows overlapping feature windows; identifiers that encode an outcome because they were assigned after it.

Clinical research. Diagnoses back-coded to admission, problem lists stored as mutable fields with no valid-time column, and cohort entry criteria that only patients who survived long enough could ever satisfy.

Incident investigation. Establishing what an operator could see on the panel is a different exercise from listing what the inquiry later determined, and the two are routinely merged into a single timeline.

Law and regulation. Patent obviousness, negligence, and the reasonableness of a supervisory decision are each judged against a state of knowledge at a date, by decision-makers who already know how it turned out.

Archival and historical scholarship. A term is read in the sense it only later acquired; a document is filed under a category invented afterwards; a correspondent's meaning is fixed by what the correspondence turned out to lead to.

Operational data systems. Overwrite-in-place storage makes "what did we believe on Tuesday" unanswerable, so every Tuesday-dated reconstruction is silently assembled out of Friday's beliefs.

Across all of them the load-bearing pair is identical: an index naming a moment, and an input whose reach began later. ^[4]

Clarity¶

The confusion this prime dissolves is a durable cross-purpose argument in which two people evaluate different properties and each takes the other for obtuse. One defends an artifact by defending its content: the number is right, the diagnosis was correct, the cause really was the seal. The other attacks it by attacking its construction: nobody could have known that then. Both can be entirely correct, because accuracy and availability are independent, and without a name for the second the disagreement has nowhere to come to rest. Naming past-state contamination gives the critic a charge that does not require exhibiting an error, and takes from the defender a reply — "but it is true" — that was never responsive to the charge. ^[5]

It also separates two failures that feel identical from the inside: an analysis that was too optimistic, and an analysis that was impossible. The first is a calibration problem with calibration remedies — shrink the estimate, widen the interval, penalise the fit. The second is a construction problem, and no amount of hedging repairs it, because the estimator was never running on the inputs it claimed to run on.

Manages Complexity¶

The thing you stop tracking is content. A naive audit of a historical claim proceeds fact by fact: for every assertion in the artifact, decide whether it was knowable then. That scales with the size of the artifact, demands domain expertise at every line, and yields judgments nobody else can reproduce. This prime replaces it with a channel audit that scales with the number of inputs instead of the number of claims — enumerate the sources, stamp each with an availability time, and ask which of them the construction could physically reach. A pipeline with three dependencies has three questions to answer no matter how many rows or pages it emits. ^[1]

The second thing you stop tracking is the individual result. Once a construction is time-respecting by design — an expanding window, a frozen snapshot, a blinded reader — the question does not have to be reopened for each output it produces. Scrutiny moves from the artifact to the apparatus, which is why the remedies here are infrastructural rather than analytical, and why an organisation that solves this tends to solve it once.

Abstract Reasoning¶

The prime licenses a diagnostic that runs with almost no subject-matter knowledge. Begin by naming t, since a contamination claim is unstatable without an index, and a surprising number of disputes end at this step because the artifact turns out not to be indexed to any particular moment and so cannot be accused of misrepresenting one. Next ask what the artifact asserts: if it speaks from the present about the past, stop; if it offers itself as something a contemporary could have produced, continue. Then enumerate the inputs and stamp each with an availability time rather than a content date, which is where most audits fail, because a restated quarterly figure, a back-coded diagnosis, and a corrected transcript all wear the label of the period they describe. Then hunt channels rather than facts, since the late item is usually invisible in the output while the mechanisms that could carry one — mutable storage, joins against live tables, random partitions, readers who saw the ending — are enumerable. Finally run the ablation: rebuild with the post-t inputs withheld, and let the size of the divergence be the measurement. ^[4]

Two further inferences are worth stating on their own. Excellent retrospective performance paired with ordinary prospective performance is evidence of contamination even before any channel has been found, because few other defects produce that particular gap. And a reconstruction that could not have been produced at t is uninformative about what a contemporary would have done, which means it cannot support the counterfactual that most such reconstructions are commissioned to support.

Knowledge Transfer¶

Four things travel intact. The two-clock structure travels, since any substrate able to index a fact both to a subject time and to an arrival time can host the pattern. The as-of claim travels, because a schema field asserts historical validity as effectively as a sentence does. The counterfactual test travels, withholding-and-rerunning being available to a pipeline, a re-reading panel, and a court excluding evidence alike. And the repair family travels: freezing, blinding, and splitting by time are the same three moves wherever the pattern appears. ^[4]

Four things do not. The threshold of availability is local — legally discoverable, physically present, actually read, and cheaply computable are four different lines, and each field draws its own. The granularity of the index is local: a microsecond in market microstructure, a fiscal quarter in accounting, a decade in historiography. Severity is local, the same structural violation being fatal in one setting and cosmetic in the next.

Most consequentially, reparability does not transfer. Where the channel is a data dependency it can be cut retroactively — rebuild the table from the audit log, refit the scaler on the training fold, and the contaminated artifact is replaced by its honest twin. Where the channel runs through a person who has read the outcome it cannot be cut at all, because a reader cannot be unblinded backwards; the only remedy is a different reader working from a redacted record. Human-substrate contamination is therefore a scheduling problem that must be solved before the fact, while machine-substrate contamination is a rebuild that can be solved after it.

Examples¶

Formal/abstract¶

Take a real-valued series and a rule that must produce, at each index t, a decision from what is available by t. In stochastic-process terms the requirement is adaptedness: the action at t must be measurable with respect to the filtration F_t, the sigma-algebra generated by everything observed up to and including t. Adaptedness is the formal statement of this prime's constraint, and its violation is the prime.

Now standardise the series before splitting it, subtracting the full-sample mean and dividing by the full-sample standard deviation, then fit and evaluate a predictor on the standardised values using a strictly chronological split. Every visible precaution is in place: the training block precedes the test block, no test label is touched during fitting, and the partition is not random. But each standardised value is a function of the entire path, including every observation after t, so no such value is F_t-measurable and the predictor operating on them is not an adapted rule. The leak is two scalars wide. It bites hardest where the test block contains a level shift or a change of volatility regime, because the shared scaler has already told the training block that the shift is coming.

Mapped back: the target time is t and the target-time information set is F_t; the late items are the observations after t that entered the two summary statistics; the artifact is the evaluation curve, which asserts as-of validity at every point by construction; the backward channel is the pair of statistics computed once over everything; and the repair is an expanding-window scaler refit at each index, whose divergence from the original result measures the contamination in the units of the thing being claimed. ^[2]

Applied/industry¶

A hospital builds an early-warning model to flag sepsis from what the record contains at hour zero of an encounter, validates it retrospectively at an area under the curve of 0.95, deploys it, and watches it fire late and often on the wrong patients.

Three data channels account for most of the gap, and none is visible in the model. First, the problem list in the warehouse is a mutable field carrying a single row timestamp rather than a valid-time column, so a diagnosis a clinician enters on day three appears in the hour-zero extract. Second, several features are derived from the encounter's final coded diagnoses and its discharge summary, both authored after the outcome they are being used to anticipate. Third, an order for serum lactate is itself a feature; the order exists because a clinician already suspected sepsis, and the source table stamps it with the resulted time rather than the ordered time, placing evidence of the suspicion earlier than the suspicion.

The human validation is compromised by a fourth channel. The abstractors who confirmed that sepsis "was identifiable at hour zero" read the complete chart, discharge summary included, before rating it. Their inter-rater agreement was excellent, and excellent agreement among readers who all know the ending measures the strength of the shared channel rather than the difficulty of the judgment. ^[5]

Mapped back: the target time is hour zero; the information set is what the record physically held then; the late items are the day-three code, the discharge summary, and the ordering clinician's suspicion; the artifacts are both the model and the adjudication that certified it; and the channels are three mutable or misaligned timestamps plus one unblinded reader. The repair splits by substrate exactly as the pattern predicts — the tables can be rebuilt bitemporally from the audit log and the model refit on that reconstruction, but the abstractors cannot be un-shown the ending, so their half of the validation must be redone by different readers working from an hour-zero redaction. Deployment was the ablation nobody scheduled, and the fall from 0.95 to the field result is the contamination read off in the open. ^[2]

Structural Tensions¶

T1 — Invisible until something diverges. Contamination leaves no trace in the artifact it damages. A leaked backtest and an honest one are the same shape of curve, and the leaked one is prettier. Discovery therefore waits on a case the artifact has genuinely never seen — a deployment, a live quarter, a fresh cohort — and until such a case arrives the only available signal is suspicion. The defect is thus systematically found late: after the artifact has been trusted, published, or acted on, and usually after the people who built it have moved on.

T2 — The price of an honest as-of view. Reconstructing what a system knew at a past moment means never overwriting anything: two timestamps on every fact, restatement vintages retained indefinitely, storage that grows with edits rather than with entities, and joins that must carry a date. Reusing the current table costs nothing and gives the right answer most of the time. The discipline is paid continuously, by everyone, against a failure that is rare, delayed, and attributable to no one in particular — close to the worst possible incentive shape for infrastructure.

T3 — Contamination versus legitimate history-dependence. A system whose present state depends on its past is not thereby contaminated; that describes most systems. The violation requires a claim of as-of validity, and claims are pitched at a level of description the author chooses. Describe an artifact loosely — our current understanding of the incident — and no boundary was ever asserted. Describe it tightly — what the controller could see at 03:14 — and the same document becomes indictable. Classification therefore rests on a reading the abstraction itself does not fix.

T4 — Green because of what was lying around. A validation that passes can pass on merit or on access, and the two are indistinguishable in the report. Suites sharing a fixture, folds sharing a scaler, cohorts sharing an adjudicator, and environments carrying state between runs all yield results whose greenness is partly a property of the harness. The stronger the reuse — and reuse is exactly what makes a large test estate affordable — the more of the result belongs to the apparatus rather than to the thing nominally under test.

T5 — Where availability actually ends. The prime needs a line between what was reachable at t and what was not, and no substrate supplies one. A filing lodged but unread, a figure derivable from published data by an analysis nobody ran, a rumour circulating on a trading floor, a lab value resulted but never looked at: each is available under one reasonable definition and unavailable under another. Every enforcement regime has to legislate the line itself, and one artifact can be clean under one convention and contaminated under the next.

T6 — Disclosure repairs the defect, not the reading. Because the violation requires an as-of claim, an artifact can be cured by relabelling: state that it speaks from the present and the structure is satisfied without a single input changing. This is the honest move and also the cheap one. Readers go on drawing the contemporaneous inference the layout invites, since a timeline still reads as a timeline, so the remedy that costs nothing ends up protecting the author considerably better than it protects the audience.

Structural–Framed Character¶

Past-State Contamination sits at the structural end of the structural–framed spectrum, graded structural with an aggregate of 0.00 and all five criteria at zero. The relation is temporal and spare: a state, belief, prediction, dataset, or decision indexed to an earlier time t; evidence that became available only after t; a reconstruction claiming to represent the state as of t that this evidence influenced; and no preserved or disclosed provenance boundary. The test is counterfactual — the artifact differs from one built solely with information available at t.

The criterion carrying the most weight is evaluative load, and despite the word contamination it reads zero. The defect concerns availability timing, not truth: the later information may be entirely correct and the contamination still holds, while retrospective explanation may legitimately use later evidence if it claims no contemporaneous knowability. What is named is a boundary violation in the artifact, closed mechanically by an immutable snapshot, blinded reconstruction, or time-respecting split.

Human-practice-bound reads zero because the relation holds between an information cutoff and an artifact: a validation pipeline instantiates it as fully as a chart reviewer. Institutional origin reads zero: the discipline-bound names are narrower, hindsight bias being the psychological sense that an outcome was predictable, look-ahead bias the quantitative-testing subtype, data leakage a crossing of other boundaries. Vocabulary travels at zero, the load-bearing terms being target time and information cutoff. Import-vs-recognize is recognition.

Structural at zero means the prime applies wherever a claim is indexed to a time, with one diagnostic: ask what was knowable at t.

Substrate Independence¶

Past-State Contamination is about as substrate-independent as a prime can be — composite 5 / 5 on the substrate-independence scale. The relation is stated over two clocks and nothing else: an artifact indexed to a target time and asserted valid as of it, an item whose availability begins strictly after that index, and a channel carrying the item in with no provenance boundary preserved. Because the definition mentions no medium, one counterfactual test — would the artifact have come out the same with the late items withheld — applies unchanged to trading backtests, overwritten operational databases, clinical chart abstraction, incident investigation, archival reconstruction, and machine-learning validation splits. Abstraction, spread, and literal instances line up, and the detection method travels with the pattern instead of being rebuilt in each field.

Composite substrate independence — 5 / 5
Domain breadth — 5 / 5
Structural abstraction — 5 / 5
Transfer evidence — 4 / 5

Relationships to Other Abstractions¶

Current abstraction Past-State Contamination Prime

Parents (1) — more general patterns this builds on

Past-State Contamination is a decomposition of Temporal Dynamics Prime

Removing domain vocabulary leaves an ordered availability relation in which later evidence crosses into an artifact asserted to represent an earlier state.

Children (2) — more specific cases that build on this

Hindsight Bias Domain-specific is a kind of Past-State Contamination

Hindsight Bias is the human-memory species in which outcome knowledge contaminates reconstruction of what was known or foreseeable before the outcome.
Data Leakage Prime is a kind of, conditional Past-State Contamination

Time leakage is the model-evaluation species in which future or held-out information enters fitting, feature construction, calibration, or validation.

Hierarchy path (1) — routes to 1 parentless root

Past-State Contamination → Temporal Dynamics → Time

Neighborhood in Abstraction Space¶

Past-State Contamination sits in a sparse region of abstraction space (60^th percentile for distinctiveness): few abstractions share its structure, so a faithful description tends to retrieve it precisely rather than landing on a neighbor.

Family — Reading Order From Structure (7 primes)

Nearest neighbors

Computed from structural-signature embeddings · 2026-09-10

Not to Be Confused With¶

Past-state contamination must first be separated from Path Dependence, the neighbour it most often collapses into. Path dependence holds that where a system is now was determined by the particular sequence it travelled, so that early and possibly arbitrary choices stay load-bearing long after the conditions producing them have gone. That is a claim about causation running forward: history shapes the present, legitimately and in the open. Past-state contamination is a claim about representation running backward: an account of an earlier moment has been shaped by information that did not yet exist. A locked-in keyboard layout is path-dependent and entirely honest about it; a memo dated before that layout was chosen which cites its market share is contaminated.

Hysteresis sharpens the same boundary in a system that holds no representations at all. A hysteretic material's current magnetisation depends on the sequence of fields it has passed through, so state alone does not determine response — the path is needed. None of this is a defect, and there is no artifact making a claim that could be violated; the memory is physical, disclosed by the loop itself, and a model omitting it is simply wrong. Contamination requires an assertion of validity as of a time, which a material never makes. Where hysteresis gets misdiagnosed as contamination, the error runs the other way: genuine history-dependence is treated as a leak to be scrubbed rather than a mechanism to be modelled.

Data Leakage is the closest relation and the most useful to place precisely, being a species rather than a synonym. Leakage names any crossing of a boundary meant to separate what a model may use from what it may not, and many of those boundaries are not temporal — a patient appearing in both folds, an identifier that encodes the label, a duplicate row spanning the split. Where the violated boundary happens to be a clock, leakage is this prime realised inside a fitting-and-evaluation pipeline. The genus is wider in substrate, covering chart abstractions, incident timelines, and judicial findings that involve no model at all, and narrower in the boundaries it recognises, since it only ever cares about time.

Caching and its failure mode run in the opposite direction, and the shared vocabulary of staleness invites confusion. A stale cache serves an earlier state into the present: the value was true at some past moment and is being presented as current. This prime serves a later state into the past: the value is true now and is being presented as contemporaneous. Both are provenance failures across a time boundary, and knowing which way the arrow points determines the fix, since a cache is repaired by invalidation and refresh while a contaminated reconstruction is repaired by freezing and withholding. Confusing the two produces exactly the wrong intervention.

Idempotence concerns residue rather than reach. An operation is idempotent when applying it twice leaves what applying it once left, which fails when a run deposits state that changes the next one. That is prior-run contamination of a present execution — again the reverse arrow. The connection worth keeping is that both defects are found with the same instrument, re-running from a clean start and comparing, even though "clean" means a fresh environment in one case and a truncated information set in the other.

Bayesian Updating is what contamination is most often mistaken for by its defenders. Conditioning on new evidence to form a new posterior is not merely permitted but obligatory, and it carries no falsehood, because the posterior is asserted for now. The violation appears only when an updated belief is presented as one that had been held before its evidence arrived. Updating changes the belief and honours the index; contamination changes the belief and misstates the index.

Anachronism is the surface-level cousin. An anachronism is an item placed in a period it does not belong to, and it is generally visible on inspection — a wristwatch in a Roman scene. This prime is defined by a channel and a counterfactual rather than by an out-of-place item, and its characteristic case leaves nothing out of place at all, because the late information is perfectly appropriate to the period it has been slipped into. Every anachronism in a document claiming period authenticity is contamination; almost no contamination is an anachronism.

Finally, Versioning, Provenance, and Blinding are remedies rather than rivals, and reading them as neighbours misplaces them. Versioning preserves the second clock, provenance records the custody chain that makes that clock trustworthy, and blinding severs the channel at a human reader. Each is infrastructure that either prevents the pattern or makes it diagnosable afterwards; none is a competing account of what went wrong.

Solution Archetypes¶

No catalogued solution archetypes reference this prime yet.

Notes¶

(Canonical first draft from the adjudicated missing-node gate. De-templated and re-authored in Claude house style on 2026-08-28: the scope block that had been pasted across What It Is Not, Clarity and Not to Be Confused With was removed and each of those sections rewritten to its own purpose, the duplicated bullet list was deleted, the two stock tensions were replaced with six specific to this prime, the Broad Use paragraph that had been copied verbatim into Knowledge Transfer and Substrate Independence was rewritten, the stock openers were cut from Manages Complexity, Clarity, Abstract Reasoning and Knowledge Transfer, and Examples was given two worked cases in place of a restated Core Idea. Structural–Framed Character and Substrate Independence are carried across unchanged from the generated draft. Independent citation review remains outstanding; no citations have been fabricated, and all fifteen FACT anchors are now web-verified (Pass 3, 2026-08-28).)

Two points that did not fit the body. The name reads in both directions and only one of them is this prime: contamination of a past-state account by later information is the canonical sense, while residue of an earlier state leaking into a supposedly fresh present belongs to caching and idempotence. The alias set — as-of contamination, future-information contamination — is worth quoting whenever that ambiguity bites.

The second point is evidential. Contamination is a fact about a construction process, and artifacts almost never carry enough process record to settle it, so most confident historical attributions rest on inference from a performance gap rather than on demonstration of a channel. That asymmetry argues for spending effort on prevention, which is cheap and verifiable at build time, rather than on detection, which is neither.

References¶

[1] Kaufman, Shachar, Saharon Rosset, Claudia Perlich, and Ori Stitelman. "Leakage in Data Mining: Formulation, Detection, and Avoidance". ACM Transactions on Knowledge Discovery from Data, 2012. Formalises leakage as the entry of information illegitimate with respect to a target, enumerates the channels it arrives through, and prescribes a data-management discipline — learn-predict separation — that closes them without anyone adjudicating the content of what crossed. registry ↩a ↩b ↩c ↩d ↩e ↩f

[2] Kapoor, Sayash, and Arvind Narayanan. "Leakage and the Reproducibility Crisis in Machine-Learning-Based Science". Patterns, 2023. Surveys leakage across hundreds of papers in many fields, finds it produced by ordinary convenience rather than intent, catalogues preprocessing fitted across the whole dataset and temporal leakage among its types, and shows reported performance collapsing once the leak is closed. registry ↩a ↩b ↩c

[3] Fama, Eugene F. "Efficient Capital Markets: A Review of Theory and Empirical Work". The Journal of Finance, 1970. Shows a field legislating its own availability line, defining weak, semi-strong and strong information sets so that what counted as reachable at a date becomes a stated convention rather than a fact the data supply. registry ↩

[4] Snodgrass, Richard T. Developing Time-Oriented Database Applications in SQL. Morgan Kaufmann, 2000. Establishes the bitemporal separation of valid time from transaction time, why overwrite-in-place storage makes prior belief states unrecoverable, and why a content date is not an availability time. registry ↩a ↩b ↩c

[5] Fischhoff, Baruch. "Hindsight Is Not Equal to Foresight: The Effect of Outcome Knowledge on Judgment under Uncertainty". Journal of Experimental Psychology: Human Perception and Performance, 1975. Demonstrates that knowing the outcome systematically alters judgements of what was knowable beforehand, and that judges do not detect the effect in themselves. registry ↩a ↩b