Skip to content

Reproducibility Protocol

Make methods, data, assumptions, and environments explicit enough that results can be repeated or checked.

The Diagnostic Story

Symptom: A result that is driving consequential action cannot be recreated: the exact data version is unclear, the parameter choices live in someone's memory, the tool environment has drifted, or the undocumented judgment calls that shaped the output cannot be recovered. Audit requests become scrambles. When the result needs to be rerun, updated, or challenged, there is no path back to the conditions that produced it.

Pivot: Define what must be reproducible and preserve the result-producing path: versioned inputs, explicit methods and parameters, documented assumptions, environment specifications, and output references — then test the preservation by actually attempting an independent rerun, replication, or audit check rather than assuming documentation is sufficient.

Resolution: Results can be verified, rerun, or audited by someone who was not present when they were produced. Version drift, tacit operator dependency, and invisible judgment paths are replaced by a preserved production path that supports accountability, learning, and maintenance.

Reach for this when you hear…

[research science] “I can't reproduce my own results from six months ago because I don't remember which preprocessing script I actually ran.”

[regulatory compliance] “The auditor wants to see exactly what data and logic produced that number and we have no way to show them.”

[ML engineering] “The model is in production but we can't retrain it because nobody saved the feature pipeline that built the training set.”

Mechanisms / Implementations

  • Reproducible Research Package (reproducible_research_package): This is a artifact that implements the archetype by preserving or testing part of the result-producing path.
  • Version-Controlled Analysis (version_controlled_analysis): This is a workflow that implements the archetype by preserving or testing part of the result-producing path.
  • Protocol Documentation (protocol_documentation): This is a document that implements the archetype by preserving or testing part of the result-producing path.
  • Lab Notebook Record (lab_notebook_record): This is a document that implements the archetype by preserving or testing part of the result-producing path.
  • Workflow Script or Pipeline (workflow_script_or_pipeline): This is a software_or_tool that implements the archetype by preserving or testing part of the result-producing path.
  • Containerized Environment Snapshot (containerized_environment_snapshot): This is a software_or_tool that implements the archetype by preserving or testing part of the result-producing path.
  • Audit Trail (audit_trail): This is a artifact that implements the archetype by preserving or testing part of the result-producing path.
  • Decision Log (decision_log): This is a document that implements the archetype by preserving or testing part of the result-producing path.
  • Replication Package (replication_package): This is a artifact that implements the archetype by preserving or testing part of the result-producing path.
  • Rerun Checklist (rerun_checklist): This is a checklist that implements the archetype by preserving or testing part of the result-producing path.

Abstractions this archetype builds on — directly (a source ingredient) or as a related pattern. Links follow the typed catalog namespace.

Built directly on (3)

Also references 4 related abstractions

Variants

Narrower or domain-specific specializations that share this archetype's core structure. Recognized variants are established; candidate variants are provisional.

Computational Reproducibility Package · implementation variant · recognized

Preserves code, data, dependencies, parameters, and expected outputs so a computational result can be rerun.

Independent Replication Protocol · governance variant · promote to full archetype candidate

Structures an independent person, team, site, instrument, or dataset to repeat the core finding or output with enough separation from the original producer.

Operational Replay Protocol · domain variant · recognized

Preserves the operational context and event sequence needed to replay, diagnose, or repeat a production process or incident response.

Decision Reconstruction Protocol · governance variant · recognized

Preserves evidence, alternatives, assumptions, authorities, and reasons so a consequential decision can be reconstructed and audited.