Reproducibility Protocol¶
Make methods, data, assumptions, and environments explicit enough that results can be repeated or checked.
The Diagnostic Story¶
Symptom: A result that is driving consequential action cannot be recreated: the exact data version is unclear, the parameter choices live in someone's memory, the tool environment has drifted, or the undocumented judgment calls that shaped the output cannot be recovered. Audit requests become scrambles. When the result needs to be rerun, updated, or challenged, there is no path back to the conditions that produced it.
Pivot: Define what must be reproducible and preserve the result-producing path: versioned inputs, explicit methods and parameters, documented assumptions, environment specifications, and output references — then test the preservation by actually attempting an independent rerun, replication, or audit check rather than assuming documentation is sufficient.
Resolution: Results can be verified, rerun, or audited by someone who was not present when they were produced. Version drift, tacit operator dependency, and invisible judgment paths are replaced by a preserved production path that supports accountability, learning, and maintenance.
Reach for this when you hear…¶
[research science] “I can't reproduce my own results from six months ago because I don't remember which preprocessing script I actually ran.”
[regulatory compliance] “The auditor wants to see exactly what data and logic produced that number and we have no way to show them.”
[ML engineering] “The model is in production but we can't retrain it because nobody saved the feature pipeline that built the training set.”
When This Archetype Applies¶
Complete catalog groundingAt least one sufficient condition set is fully represented by existing primes or domain-specific abstractions.
Diagnostic problem
A result, decision, analysis, experiment, operational run, or artifact is treated as reliable even though the information needed to recreate or audit it has not been preserved. The result becomes dependent on memory, tacit context, fragile tooling, undocumented choices, or inaccessible inputs.
What this problem means
The structural problem is an invisible production path. A team produces a result, but the details that made it possible are scattered across memory, local files, undocumented parameter choices, tacit conventions, expired dependencies, shifting data versions, and unrecorded assumptions. Later, the result cannot be recreated or audited without guesswork.
This failure often appears as an audit scramble: people reconstruct a plausible story after the fact rather than presenting a preserved path. It also appears as version drift, where reruns silently use different inputs, code, tools, instruments, or decision criteria.
Show the applicability expression
Applicability expression3 distinct conditions
groundedpartly groundedopen
3 conditions, all required.
3Required in every casenumbered 1–3
These hold no matter which pattern applies.
Context-dependent output · grounded
The output depends on specific data, tools, parameters, assumptions, roles, environmental conditions, or handoffs.
A team produces a result, but the details that made it possible are scattered across memory, local files, undocumented parameter choices, tacit conventions, expired dependencies, shifting data versions, and unrecorded assumptions. The narrower requirement in this condition set is: The output depends on specific data, tools, parameters, assumptions, roles, environmental conditions, or handoffs.
Future rerun need · grounded
Another person, team, regulator, maintainer, reviewer, or future version of the same team may need to verify or rerun the result.
A team produces a result, but the details that made it possible are scattered across memory, local files, undocumented parameter choices, tacit conventions, expired dependencies, shifting data versions, and unrecorded assumptions. The narrower requirement in this condition set is: Another person, team, regulator, maintainer, reviewer, or future version of the same team may need to verify or rerun the result.
Consequential result reuse · grounded
The result may be challenged, replicated, scaled, transferred, automated, or used as precedent.
It is especially useful when results may be reviewed by others, challenged by stakeholders, reused by future teams, scaled to new contexts, or used as evidence for consequential decisions. The narrower requirement in this condition set is: The result may be challenged, replicated, scaled, transferred, automated, or used as precedent.
Other requirements and context (2)
Why these sit outside the expression
Supporting context — it may accompany or help interpret the situation, but it is not a load-bearing condition in a sufficient diagnostic set.
Supporting contextA claim, report, model, decision, experiment, production run, or operational procedure will influence consequential action.
Use this archetype when a finding, decision, analysis, production run, operational response, model output, or report will matter beyond the moment of production. In this archetype, the relevant contextual consideration is: A claim, report, model, decision, experiment, production run, or operational procedure will influence consequential action. It helps interpret the situation or strengthens the practical case for examining the archetype.
Supporting contextErrors, drift, undocumented changes, or tacit know-how would be costly if discovered late.
The result becomes dependent on memory, tacit context, fragile tooling, undocumented choices, or inaccessible inputs. In this archetype, the relevant contextual consideration is: Errors, drift, undocumented changes, or tacit know-how would be costly if discovered late. It helps interpret the situation or strengthens the practical case for examining the archetype.
Coverage
3 of 3 conditions grounded.
Mechanisms / Implementations¶
- Reproducible Research Package (
reproducible_research_package): This is a artifact that implements the archetype by preserving or testing part of the result-producing path. - Version-Controlled Analysis (
version_controlled_analysis): This is a workflow that implements the archetype by preserving or testing part of the result-producing path. - Protocol Documentation (
protocol_documentation): This is a document that implements the archetype by preserving or testing part of the result-producing path. - Lab Notebook Record (
lab_notebook_record): This is a document that implements the archetype by preserving or testing part of the result-producing path. - Workflow Script or Pipeline (
workflow_script_or_pipeline): This is a software_or_tool that implements the archetype by preserving or testing part of the result-producing path. - Containerized Environment Snapshot (
containerized_environment_snapshot): This is a software_or_tool that implements the archetype by preserving or testing part of the result-producing path. - Audit Trail (
audit_trail): This is a artifact that implements the archetype by preserving or testing part of the result-producing path. - Decision Log (
decision_log): This is a document that implements the archetype by preserving or testing part of the result-producing path. - Replication Package (
replication_package): This is a artifact that implements the archetype by preserving or testing part of the result-producing path. - Rerun Checklist (
rerun_checklist): This is a checklist that implements the archetype by preserving or testing part of the result-producing path.
- Audit Trail
- Containerized Environment Snapshot: Captures software, dependency, and runtime context so computational behavior can be rerun under a known environment.
- Decision Log: Captures each significant decision as a linked record — its rationale, the alternatives weighed, who approved it, and the artifacts it affects — so a choice can later be traced back to why it was made and forward to what it touched.
- Lab Notebook Record: Records experimental conditions, materials, observations, deviations, and interpretive notes so later teams can reconstruct the work.
- Protocol Documentation: Describes the ordered method, required inputs, assumptions, roles, and output checks that allow a process or analysis to be repeated.
- Replication Package: Packages enough material for an outside person or team to repeat, verify, or challenge the original result.
- Reproducible Research Package: Bundles data, code, methods, documentation, and expected outputs so a scientific or analytic result can be rerun or inspected.
- Rerun Checklist: Provides a lightweight confirmation list for rerunning the result path and comparing outputs against the reference.
- Version-Controlled Analysis: Uses a version-control system to preserve changes to code, data-processing scripts, notebooks, parameters, and documentation.
- Workflow Script or Pipeline: Automates the steps that transform inputs into outputs, reducing hidden manual variation and making reruns observable.
Related Abstractions¶
Abstractions this archetype builds on — directly (a source ingredient) or as a related pattern. Links follow the typed catalog namespace.
Built directly on (3)
- Data Integrity: Accuracy and consistency preserved.
- Reproducibility & Replicability: Repeatable results.
- Versioning: Tracks incremental changes over time.
Also references 4 related abstractions
- Accountability: Responsibility for actions.
- Interoperability: Systems function together.
- Observability: Infer internal state externally.
- Transparency: Open processes.
Variants¶
Narrower or domain-specific specializations that share this archetype's core structure. Recognized variants are established; candidate variants are provisional.
Computational Reproducibility Package · implementation variant · recognized
Preserves code, data, dependencies, parameters, and expected outputs so a computational result can be rerun.
Independent Replication Protocol · governance variant · promote to full archetype candidate
Structures an independent person, team, site, instrument, or dataset to repeat the core finding or output with enough separation from the original producer.
Operational Replay Protocol · domain variant · recognized
Preserves the operational context and event sequence needed to replay, diagnose, or repeat a production process or incident response.
Decision Reconstruction Protocol · governance variant · recognized
Preserves evidence, alternatives, assumptions, authorities, and reasons so a consequential decision can be reconstructed and audited.
Editorial Notes¶
Problem Classification¶
Classification: Identity, Provenance & Integrity Failure → Temporal Record Lineage & Reproducibility
Problem kernel: the production path needed to reproduce a result is unrecorded
Rationale: Earliest causal condition: A result, decision, analysis, experiment, operational run, or artifact is treated as reliable even though the information needed to recreate or audit it has not been preserved. The result becomes dependent on memory, tacit context, fragile tooling, undocumented choices, or inaccessible inputs.
Independent corroboration: The earliest necessary condition in the frozen evidence is: A result, decision, analysis, experiment, operational run, or artifact is treated as reliable even though the information needed to recreate or audit it has not been preserved. That is a temporal record lineage and reproducibility problem because Overwriting, delayed capture, or unversioned accumulation erases how the present artifact, decision, result, or collective memory formed and prevents faithful reconstruction or replay.
Review outcome: Independent reviewer agreement; high confidence.