Reproducibility & Replicability

one_liner: Repeatable results.

core_idea:
Reproducibility and replicability constitute the repeat-the-study-to-confirm-the-finding principle that:
(1) **reproducibility and replicability are complementary concepts about independent verification of scientific findings: *reproducibility* refers to obtaining consistent results from the same data using the same analytic procedures — the "computational" sense that a well-documented analysis can be rerun by a third party with the original data to yield identical results — while *replicability* refers to obtaining consistent results from new data using similar methods — the "scientific" sense that an effect reported in one study can be observed again in an independent study collecting fresh data under similar conditions; the two concepts were traditionally conflated in the phrase "replication," but the 2019 National Academies of Sciences *Reproducibility and Replicability in Science* report formalized the distinction<!-- FACT-D25-136 -->[^national-academies-2019], with reproducibility treating same-data-same-analysis as the computational standard and replicability treating new-data-same-design as the scientific-confirmation standard**; both concepts rest on the philosophical commitment — rooted in Popper (falsifiability; 1934 *Logik der Forschung*), the hypothetico-deductive tradition, and the sociology of science (Merton's norms of universalism and communism) — that scientific knowledge must be independently verifiable rather than depending on singular authoritative demonstrations; the `multi_origin_equal` flag reflects genuine co-origin in experimental-design/statistics (where replication is a foundational concept for valid inference — Fisher 1935 "the greater the precision aimed at, the greater the demand for replication") and philosophy of science (Popper; Lakatos; Kitcher; contemporary philosophy-of-science analyses of replication);
(2) the concept has **several identifiable components and distinctions**: *reproducibility* (same data, same code, same result — a computational standard), *replicability* (new data, same methods, consistent result — a scientific standard), *generalizability* (effect holds under deliberately-varied conditions — a more stringent standard), *robustness* (effect holds under different analytical specifications — a sensitivity standard), *direct replication* (attempting to reproduce the original study as closely as possible), *conceptual replication* (testing the same theoretical prediction with deliberately different methods), *pre-registered replication* (replication attempt registered before conducting, with pre-specified success criteria), *many-labs replication* (coordinated multi-laboratory replication effort), *registered report* (journal commits to publication based on methodology rather than results), *replication crisis* (the empirical observation, documented across psychology, medicine, economics, and other fields, that many published findings fail to replicate; Ioannidis 2005 "Why most published research findings are false"<!-- FACT-D25-145 -->[^ioannidis-2005]; Open Science Collaboration 2015 *Science* paper reporting 36% replication rate in psychology; Camerer et al. experimental-economics replications; Begley-Ellis 2012 preclinical-cancer-research; multiple-field replication initiatives), and *reproducibility crisis* (the computational version — inability to reproduce published computational results from available code and data, as documented in Nature surveys and discipline-specific audits);
(3) the deeper logic is that **a single study, no matter how carefully conducted and analyzed, has substantial probability of producing a misleading finding — due to sampling variability, selection effects, winner's-curse from publication threshold, p-hacking, garden-of-forking-paths flexibility, and the fundamental statistical fact that published effects in significance-selected studies over-represent the high-variance tail of their sampling distributions** — and therefore independent replication is not an optional refinement but a constitutive feature of reliable scientific knowledge; the `contested_construct` flag reflects ongoing disagreement about (a) whether the "crisis" is crisis or normal scientific self-correction, (b) how to define replication success (exact-effect-size match, confidence-interval overlap, same-sign, same-sign-and-significance), (c) what replication rates should be expected given real-effect heterogeneity and statistical power, (d) which replication failures reflect original false positives versus replication false negatives (low replication-study power) versus true between-study heterogeneity in effect sizes, and (e) whether replication can ever be "exact" given inevitable context-and-population differences; the consensus position across most methodological reformers is that current scientific incentives (publication bias toward novel significant findings; career rewards for novelty over verification; under-resourcing of replication; rare data sharing) produce systematically less-replicable research than the epistemic ideal demands, and reform in multiple dimensions (pre-registration; registered reports; data-and-code sharing; replication funding; replication-positive journals; meta-science infrastructure) has been underway since the 2011-2015 reform wave;
(4) the concept appears across domains — **psychology and behavioral science** (Open Science Collaboration 2015 Reproducibility Project; Many Labs consortium; registered reports at *Royal Society Open Science*, *Cortex*, *Nature Human Behaviour*; ≈36% replication rate in psychology flagship study; individual cases including Bem ESP 2011 non-replication and priming research broadly), **medicine and biomedicine** (Begley-Ellis 2012 *Nature* report that 47 of 53 "landmark" preclinical cancer studies did not replicate; Prinz-Schuonenberg-Asadullah 2011 Bayer in-house audit showing 21% replication rate; Ioannidis 2005 foundational meta-analysis; Reproducibility Project: Cancer Biology; direct-to-human research more stringent than preclinical), **economics and econometrics** (Camerer et al. 2016 *Science* replication of 18 experimental-economics studies with 61% replication rate; Camerer et al. 2018 social-science-broad replication with 62% rate; AEA data and code availability policy since 2005; replication-in-economics journals), **political science and sociology** (multiple replication initiatives; TESS data-sharing; some high-profile non-replications including LaCour 2014 retraction), **computational and data science** (computational reproducibility distinct from scientific replicability; "reproducibility crisis" in ML where results depend on undisclosed hyperparameters, random seeds, implementation details; NeurIPS and ICML reproducibility checklists), **genetics and genomics** (early GWAS era replete with non-replicating candidate-gene studies; contemporary GWAS requires explicit replication in independent cohorts for publication), **drug development and clinical trials** (Phase 3 replication expectation for regulatory approval — typically two positive trials; FDA/EMA review of replication patterns across submitted evidence), **ecology and evolutionary biology** (replication studies more recent but growing; SCORE and SPEC projects in ecology), **physics and chemistry** (independent-laboratory confirmation of major experimental results as standard; Higgs boson 5σ evidence required independent confirmation from ATLAS and CMS; traditional physics-culture expectation of cross-laboratory reproduction), **meta-science and science-of-science** (Bak-Coleman and others using replication data to study scientific production; Meta-Research Innovation Center at Stanford; the TIER protocol; the FAIR data principles) — across these, the repeat-the-study-to-confirm principle is shared, with domain-specific replication norms, failure modes, and reform efforts.

structural_signature:
A reproducibility-and-replicability assessment exhibits the following six core role-phrases:

*the original-study artifact-and-result* — a published finding from an original study, typically a specific claim (effect exists, effect has specific magnitude and direction, association is statistically significant, pattern is observable), published in a peer-reviewed venue with sufficient detail that readers might evaluate or replicate it.

*the independent-replication attempt under matched conditions* — either the computational rerun (reproducibility, same data and code) or a new data-collection study following the original methodology (replicability, new data, same design), conducted with sufficient independence and methodological fidelity to test whether the original finding holds.

*the operational-definition of replication success* — a pre-specified criterion for what would count as successful reproduction or replication (match of computational results, same-sign significant result in new data, confidence-interval overlap, effect-size match within tolerance, or other definition used in the original study's field or replication protocol).

*the documentation-and-transparency requirement* — availability of original data and code in usable form (reproducibility requirement), and availability of original methodology documentation sufficient to re-implement the study (replicability requirement), enabling independent verification.

*the power-and-statistical-adequacy consideration* — replication sample size must be adequate to detect the original effect size with reasonable probability (typically ≥80% power to detect the original reported effect, or ≥80% power to detect a smaller plausible effect if accounting for winner's-curse inflation).

*the result-interpretation and aggregation framework* — analysis of reproduction/replication results against the original finding using pre-specified success criteria; interpretation of outcomes (successful replication strengthens evidence; failed replication triggers diagnostic questions about original false positive vs. replication false negative vs. population heterogeneity vs. method differences); meta-analytic synthesis and aggregation across multiple replication attempts when available.

When these elements are in place, replication functions as the self-correction mechanism of science; when they are absent (no data-and-code sharing; inadequate methodology documentation; under-powered replications; ambiguous success criteria), replication efforts produce inconclusive results and cannot correct the scientific record.

what_it_is_not:
- Not **redundant with statistical significance** — a single study reaching statistical significance does not establish the finding; significance controls the Type I error rate for that test, not the reliability of the finding across studies. Replication provides a different and stronger evidential layer.
- Not **a guarantee of truth even when successful** — multiple consistent findings can all be produced by shared methodological flaws, publication bias, measurement-instrument artifacts, or stable-but-invalid confounders. Replication is necessary but not sufficient for confident inference.
- Not **impossible in principle** — some philosophical discussions have argued that no replication is "exact" because population, time, and context always differ, concluding that replication is a meaningless ideal. The pragmatic response (embraced by most of the replication-reform movement) is that "close-enough" replication is possible and informative, and that deliberate variation can probe generalizability rather than undermining replication as a concept.
- Not **solely about fraud detection** — replication failures can arise from many causes (chance; p-hacking; selection bias; population heterogeneity; methodological artifacts; measurement differences; true between-study effect-size variation) with fraud representing a tiny fraction. Treating replication primarily as fraud detection misframes the function.
- Not **equivalent across fields** — conventions for what counts as replication differ substantially across disciplines. Physics culture expects independent-laboratory confirmation as a routine standard; psychology historically treated single-study findings as sufficient until the replication-crisis wave; economics relies on applied-policy evidence that is hard to replicate without natural-experiment re-emergence. Field-specific norms shape what replication looks like.
- Not **addressable only by replicators** — the conditions for replicability are set by original-study authors (methodology documentation; data sharing; pre-registration; analysis transparency) and by the publication and funding systems (rewarding replication efforts; supporting data infrastructure). Replication success is a systemic property, not solely a replicator's achievement.
- Not **meaningfully assessed by a single replication attempt** — failed replication can occur due to replication-study shortcomings (low power; design error; population differences) rather than original-study errors. Multiple replication attempts, preferably with different methods and contexts, provide the evidential base for confident conclusions.
- Not **the same as "robustness" in the sense of analytic-decision sensitivity** — robustness checks (varying analytic specifications within a study) are useful but distinct from replication (new data). Both are important for reliable inference but operate at different levels.
- Not **orthogonal to statistical power** — low-powered original studies produce inflated effect estimates (winner's curse); adequately-powered replications then produce smaller estimates, often failing to reach significance even when the true effect exists. Power considerations are central to replication interpretation.
- Not **a simple binary outcome** — "successful" and "failed" replication categories oversimplify. Effects can replicate in sign but not magnitude; in some populations but not others; statistically significantly but with smaller effects than original; or the reverse. Continuous reporting of replication results (effect-size estimate, CI, power, comparison to original CI) is more informative than binary classification.

broad_use:
- **Psychology and behavioral science (canonical replication-crisis context)**: Bem 2011 ESP paper published in a major journal triggered methodological concerns. Open Science Collaboration 2015 Reproducibility Project: Psychology replicated 100 published studies with ≈36% replication rate (varying by replication definition); average replication effect size was about half the original. Many Labs Project (Klein et al. 2014, 2018, 2022) conducted coordinated multi-laboratory replications with more consistent patterns than single-lab replications. Registered reports at *Royal Society Open Science*, *Cortex*, *Nature Human Behaviour*, *Psychological Science* commit to publication based on pre-registered methodology rather than results. Pre-registration infrastructure (AsPredicted, OSF) became widespread. Power-analysis standards for sample size improved. Priming research, ego-depletion, and other subfield-specific replication failures prompted theoretical reassessment.
- **Medicine and biomedicine**: Ioannidis 2005 *PLoS Medicine* "Why most published research findings are false" used prior-probability reasoning to argue that most published findings are likely false under plausible assumptions. Begley-Ellis 2012 reported that only 6 of 53 "landmark" preclinical oncology studies replicated — a 11% replication rate in a disease area with high-stakes translation. Prinz-Schuonenberg-Asadullah 2011 Bayer in-house audit showed 21% replication rate across disease areas. Reproducibility Project: Cancer Biology (2014-2021) formally replicated 50 high-profile cancer studies with mixed results. Clinical-trial replication is somewhat better (Phase 3 trials with pre-registration and regulatory oversight) but preclinical research remains concerning. FDA and EMA increasingly scrutinize replication patterns and pre-registration in approval decisions.
- **Economics and econometrics**: Camerer et al. 2016 replication of 18 experimental-economics studies found 61% replication rate. Camerer et al. 2018 *Nature Human Behaviour* social-science replication found 62% rate across 21 studies from *Nature* and *Science*. Econometric replication requires different definitions (quasi-experimental findings rely on specific natural experiments not available for re-examination; structural models depend on specific assumption-sets). AEA data and code availability policy since 2005; *American Economic Review* and others enforce replication-file availability. Credibility revolution in empirical economics (Angrist-Pischke) overlaps with replication-consciousness.
- **Political science and sociology**: Multiple replication initiatives including TESS (Time-sharing Experiments for the Social Sciences). High-profile cases include LaCour 2014 retraction. Data-sharing norms improving but uneven.
- **Computational and data science (canonical computational-reproducibility context)**: Distinct reproducibility crisis in ML — published results often depend on undisclosed hyperparameters, random seeds, data-preprocessing choices, and implementation details. Hutson 2018 reported that most ML papers cannot be reproduced from their publications alone. NeurIPS, ICML, and other venues added reproducibility checklists. MLPerf benchmarks and reproducibility challenges. "Paper with code" movement. Docker containers and cloud notebooks for computational-reproducibility support.
- **Genetics and genomics**: Early GWAS era was replete with non-replicating candidate-gene studies — small-sample studies proposed associations that failed in larger samples. Contemporary GWAS explicitly requires replication in independent cohorts for publication (typically discovery cohort plus replication cohort). The 5×10⁻⁸ genome-wide-significance threshold plus required replication has produced highly reliable GWAS findings. Replication culture in human genetics is generally strong relative to many biomedical subfields.
- **Drug development and clinical trials**: Phase 3 replication expectation for regulatory approval — typically two positive trials demonstrating efficacy before full drug approval. Regulatory reviewers scrutinize replication consistency; approval often discusses whether the second trial truly replicated the first. Pre-specified primary endpoints and statistical-analysis plans codify replication standards across the regulatory pipeline. Post-approval effectiveness studies extend the evidence base.
- **Ecology and evolutionary biology**: SPEC (Ecology Replication Project) and related efforts. Field ecology faces unique replication challenges (natural variation; site-specific ecology; long-term timescales) but meta-analytic synthesis and coordinated experimental networks partially address replication concerns. Nakagawa and colleagues have developed replication methodology for ecological contexts.
- **Physics and chemistry**: Traditional physics culture has strong replication norms. Major experimental results (Higgs boson; gravitational waves; recent cosmological measurements) require independent-laboratory confirmation. Tabletop-physics results are expected to replicate across laboratories. The replication crisis has been less prominent in physics than in biomedicine and behavioral science, partly reflecting culture and partly reflecting publication practices.
- **Meta-science and science-of-science**: Stanford Meta-Research Innovation Center (METRICS) founded by Ioannidis 2014. The Center for Open Science (COS) founded 2013. Studies of replication rates, pre-registration adoption, data-sharing rates, and science-reform impact. Bak-Coleman and others use replication data to study scientific production dynamics. The FAIR data principles (Findable, Accessible, Interoperable, Reusable) codify data-infrastructure norms supporting reproducibility.

clarity: Names the specific epistemic standards — reproducibility (same data) and replicability (new data)<!-- FACT-D25-146 -->[^national-academies-2019] — that distinguish reliable scientific findings from singular-study artifacts. Without the frame, people treat single-study findings as established knowledge, fail to distinguish computational reproducibility from scientific replicability, and miss the systematic reasons why original published findings may not replicate (publication bias, winner's curse, p-hacking, low power, selection effects). With the frame, diagnosis becomes specific: *has this finding been replicated, and under what conditions? Was the replication preregistered with pre-specified success criteria? Was the replication adequately powered to detect an effect of the size the original reported — or of a smaller plausible true effect size? Is the original study's data and code available for reproducibility checking? If replication has not been conducted, what is the baseline expectation given the field's replication-rate patterns? Is the original finding in a subfield with strong replication culture (physics, genetics, clinical trials) or weak (preclinical biology, social psychology)? Does the finding rest on a single high-powered replication or on many independent replications? How should I weight this evidence in my decisions?* The frame clarifies that scientific findings exist on a continuum from single-study-speculation to multiply-replicated-consensus, and that the evidential weight of a finding depends on where it sits on that continuum.

manages_complexity: Decomposes reliability into computational (reproducibility), scientific (replicability), methodological (robustness), and epistemic (generalizability) dimensions<!-- FACT-D25-147 -->[^national-academies-2019], each with distinct defenses and assessments. Cross-domain transfer is productive: pre-registration from clinical trials to psychology to economics; data-sharing requirements from genetics to economics to social sciences; registered reports from psychology to biomedicine; meta-analytic replication aggregation from medicine to all empirical fields; computational-reproducibility infrastructure (Docker, notebooks, cloud computing) from computer science to all computational sciences. The decomposition reveals interplay with other primes: *randomization* (#432) — proper randomization is a condition for reliable replication; *hypothesis testing* (#434) — replication aggregates across single-study testing; *statistical significance / p-value* (#435) — p-value interpretation is compromised without replication expectation; *statistical power* (#437) — under-powered original studies produce winner's curse and non-replicable findings; *selection bias* (#440) — publication bias is selection bias in literature, a primary driver of replication failure; *confounding* (#438) — unmeasured confounders differing across replication contexts produce non-replicable findings; *effect size* (#447) — replications measure whether effect sizes hold up; *sampling representativeness* (#433) — replications in different populations test generalizability.

abstract_reasoning: The analyst asks: *what is the evidential status of this finding — single study, one replication, multiple replications across contexts?<!-- FACT-D25-148 -->[^nosek-2015] What field does it come from, and what are that field's replication-rate patterns and reform maturity? Is the original study well-powered, pre-registered, with transparent data and code, or under-powered with undisclosed analytic flexibility? Is the effect magnitude consistent with what statistically-plausible winner's-curse arguments would imply for a single-study significant finding? If replication studies exist, were they pre-registered, adequately powered, and conducted with methodological fidelity to the original? If replications failed, what explains the failure — original false positive, replication false negative, between-study heterogeneity, methodological differences? How should I weight this evidence relative to singular-study findings in fields with stronger replication cultures? Is there sufficient aggregated evidence for practical-decision confidence, or is more replication needed?* Mature practice reads single-study findings with appropriate skepticism, tracks replication status of findings they rely on, supports reform through pre-registration and data-sharing, and treats scientific reliability as emerging from accumulated evidence rather than authoritative-source publication. Immature practice treats published findings as established knowledge, ignores replication status, and treats replication failure as individual-study failure rather than as systemic methodological concern.

knowledge_transfer:
| Domain | Replication-rate estimate | Primary failure-mode | Ongoing reform |
| --- | --- | --- | --- |
| Social psychology | ~25-50% (varies by definition) | Low power; flexibility | Pre-registration; registered reports |
| Cognitive psychology | ~50-65% | Effect-size inflation | Power improvements |
| Experimental economics | ~60% | Context dependence | Data-sharing mandates |
| Preclinical cancer biology | ~10-25% | Methodological diversity; bias | Biotech in-house audits |
| Clinical trials (regulatory) | High (with 2-trial rule) | Post-approval drift | Real-world evidence |
| GWAS | High (with replication-cohort rule) | Stringent thresholds work | Continuing |
| Machine learning | Low (computational reproducibility) | Undisclosed details | Reproducibility checklists |
| Physics (major results) | High | Systematic biases rare | Independent-lab verification |
| Ecology | Variable; emerging | Site-specificity; natural variation | Coordinated networks |
| Computational chemistry | Low-to-variable | Code/data availability | FAIR principles |

Across rows: the core logic — repeated independent observation as evidential standard — transfers across domains, with characteristic failure modes and reform trajectories.

formal_abstract:
The Open Science Collaboration's 2015 *Science* paper "Estimating the reproducibility of psychological science" (Science 349, aac4716) is the canonical modern replication-study case. Led by Brian Nosek at the University of Virginia and the Center for Open Science, the project coordinated a multi-year, multi-laboratory effort to conduct high-fidelity replications of 100 published studies from three top psychology journals (*Psychological Science*, *Journal of Personality and Social Psychology*, *Journal of Experimental Psychology: Learning, Memory, and Cognition*) published in 2008. The replication studies followed the original protocols as closely as possible, typically with larger sample sizes than the original (median replication sample size was 2.5 times the original, attempting to provide >80% power to detect the original effect size)<!-- FACT-D25-137 -->[^osc-2015]. Replication success was operationalized through multiple metrics: (a) did the replication produce a statistically significant result in the same direction as the original (significance-based)? (b) was the original effect size within the replication's 95% confidence interval (interval-overlap-based)? (c) did the meta-analytic combination of original and replication provide significant evidence (combined-evidence-based)? (d) did the replication's authors subjectively judge the result a successful replication (subjective)? Across these metrics, replication rates were approximately: 36% by same-direction significance; 47% by interval overlap; 68% by combined-evidence; 39% by subjective judgment<!-- FACT-D25-138 -->[^osc-2015-rates]. The mean replication effect size was approximately half the original (0.197 vs. 0.403 standardized effect), consistent with substantial winner's-curse inflation in original published effects. Replication success was correlated with original effect size (larger originals more likely to replicate); original p-value (stronger originals more likely); original effect surprising-ness (surprising findings less likely to replicate); and subfield (cognitive-psychology results replicated better than social-psychology). Individual cases included notable non-replications (some priming-research findings; some dual-process cognitive findings) and notable successful replications (many perceptual and memory findings). The paper's publication in *Science* generated massive methodological-reform impact: (i) It quantified for the first time the psychology replication problem at scale with rigorous methodology, converting anecdotal concerns into statistical evidence<!-- FACT-D25-139 -->[^osc-2015-impact]. (ii) It distinguished different definitions of replication success, showing that the estimated rate depends substantially on definition. (iii) It motivated subsequent initiatives — Reproducibility Project: Cancer Biology (biomedical analogue), Many Labs Project (multi-lab coordinated replications), Social Psychology Association Center of Excellence for Open Science. (iv) It accelerated pre-registration adoption in psychology — AsPredicted, OSF pre-registration, Registered Reports formats in multiple journals. (v) It prompted power reform — minimum-sample-size expectations and explicit power-justification requirements became standard at many psychology journals. (vi) It prompted theoretical reassessment of multiple subfields (ego depletion, priming effects, facial feedback hypothesis — all substantially re-examined post-replication). (vii) It entered the broader scientific and public discourse as a central reference for "replication crisis" claims. Subsequent methodological work extended the findings: (a) Camerer et al. 2016 (experimental economics, ~60% replication), 2018 (social-science-broad, ~62% replication)<!-- FACT-D25-140 -->[^camerer-2018]. (b) Many Labs consortium's more-systematic replications showing more reliable patterns than single-lab replications in many cases. (c) Open Science Framework's growth as infrastructure for pre-registration and data-sharing. (d) Institutional and funding-agency requirements for data-sharing and pre-registration. (e) Multiple-Aims-Score (MASS) and other aggregated-evidence approaches to meta-analytic replication synthesis. (f) Field-specific replications in economics, neuroscience, ecology, and other domains. The Open Science Collaboration's 2015 paper represents a benchmark example of the replication-as-scientific-quality-assessment concept: the explicit, large-scale, rigorous, pre-registered multi-laboratory effort to measure scientific reliability, with results that were consequential for methodological reform and that continue to shape research practice a decade later.

**Mapped back:** The OSC formal case establishes the canonical framework for replication assessment: design, operationalization of success criteria, analysis across multiple definitions, outcome interpretation (including effect-size attenuation consistent with publication-bias theory), and downstream institutional impact demonstrating how rigorous replication evidence drives methodological reform.

applied_industry:
A mid-sized consumer-finance company's data-science team discovers a promising pattern in its customer-transaction data: customers who use a specific combination of payment features appear to have substantially lower default rates on credit products than customers who do not. The original analysis, based on 18 months of transaction data covering approximately 2.1 million customer-months, identifies a statistical relationship with a hazard ratio of approximately 0.72 (28% reduction in default rates) among customers using the feature combination, p < 0.001, with effects persisting after adjustment for credit-score decile, account tenure, product mix, and demographic controls. The team is excited and proposes a marketing campaign to promote the feature combination, projecting $18M annual reduction in default costs at moderate marketing investment. Before launching, the Chief Risk Officer requests a more rigorous verification, specifically raising concerns about: (a) Whether the finding would replicate in a separate dataset and time window, (b) Whether the finding reflects causation or selection — customers who choose to use certain features may differ systematically from those who do not, and (c) Whether a proposed marketing intervention would produce the same effect as the observed passive association<!-- FACT-D25-141 -->[^munafo-2017]. The verification plan addresses all three concerns: (i) *Out-of-sample reproducibility*: Re-running the original analysis on a held-out 6-month window following the original 18-month window, with the same analytic code and procedures. Result: Hazard ratio replicates at 0.78 (95% CI 0.72-0.85) — slightly attenuated from the original 0.72 but still showing substantial association. The reproducibility check confirms the pattern is not an artifact of the specific time window. (ii) *Selection-bias diagnostic*: Analysis of customer characteristics at the time of feature-combination adoption, showing that adopters differ systematically from non-adopters on multiple pre-adoption variables (higher credit-score, longer tenure, lower income volatility, better payment history) — not all of which were in the original adjustment set. Propensity-score-adjusted analysis using expanded covariate set yields hazard ratio of 0.89 (95% CI 0.82-0.97) — a 11% reduction, substantially smaller than the naive 28% but still significant. The selection-bias diagnostic suggests that much of the observed effect reflects customer-quality selection rather than causal feature effect<!-- FACT-D25-142 -->[^simmons-nelson-simonsohn-2011]. (iii) *Causal verification via randomized promotion*: A randomized experiment is designed to promote the feature combination to a random sample of customers eligible to adopt, with a comparable control group receiving no promotion. Promotion increases feature-combination adoption from 8% to 24% among the promoted group. Intention-to-treat analysis of default rates: Hazard ratio 0.96 (95% CI 0.88-1.05) — essentially no effect. Complier-Average-Causal-Effect (using random promotion as instrument for feature adoption): 0.85 (95% CI 0.72-1.00) — some effect, but with CI including 1.0 and substantially smaller than the original 0.72 observational estimate. The three-stage verification tells a clear story: (α) The original pattern reproduces in held-out data (not a spurious artifact), but (β) selection bias explains much of the observed effect (customers adopting the feature combination were pre-selected on lower-default-risk characteristics), and (γ) randomized promotion produces much smaller causal effects than observational analysis suggested<!-- FACT-D25-143 -->[^ioannidis-2005]. The Chief Risk Officer's decision: Do not launch the originally-proposed marketing campaign; the projected $18M annual default reduction was substantially overestimated because the original observational estimate conflated selection and causal effects. However, a smaller modified marketing campaign targeting specific customer segments where the CACE remained meaningfully non-zero is approved, with pre-registered randomized expansion to test if the effect holds in broader segments. The case illustrates reproducibility-and-replicability analysis as practical application to business decision-making: computational reproducibility (same analysis on held-out data) confirms the pattern is real in the data; scientific replicability (randomized follow-up) reveals that the observational pattern's causal interpretation was substantially overestimated; business decision is made on the more-rigorous evidence base. The contrast between the $18M naive estimate and the much smaller causally-supported opportunity — and the distinction between "pattern in data" (which reproduced) and "causal effect of intervention" (which was much smaller) — makes both methodological points vivid.

**Mapped back:** The industry case demonstrates reproducibility-and-replicability reasoning in a business context with stakes: computational reproducibility (out-of-sample validation) confirms pattern persistence; selection-bias adjustment and randomized causal verification reveal that the original observational effect was substantially inflated; the decision-maker uses the more-rigorous evidence base to revise estimates and strategy, avoiding a large false-positive decision.

prose:
_(Pass B — deferred)_

### Catalog cross-references (v1, batches 02–29)

Solution archetypes that reference this prime (per `solution_archetypes/_catalog/catalog_v1_batches_02_29.yaml`):

**As source prime (1):** `proceduralization`

**As related prime (6):** `checkpoint_and_rollback`, `false_convergence_prevention`, `perturbation_testing`, `traceability_linking`, `variance_reduction`, `versioned_evolution`

notes:
Munafò et al. 2017 manifesto [^munafo-2017]<!-- FACT-D25-144 --> articulates the methodological-reform framework for reproducible science.

The `multi_origin_equal` flag is warranted<!-- FACT-D25-150 -->[^fisher-1935] — reproducibility-and-replicability has genuine co-origin in experimental-design/statistics (Fisher 1935 explicitly treated replication as central to inference — "the greater the precision aimed at, the greater the demand for replication"; replication is foundational to multi-study methodological frameworks) and in philosophy of science (Popper falsifiability framework; Lakatos research-programme methodology; Kitcher naturalistic epistemology; contemporary philosophy-of-science and science-studies analyses). The primary `origin_domain: experimental_design_statistics` reflects the operational-methodology framework; `alternate_origin_domains: [philosophy]` reflects the epistemological foundation. The `contested_construct` flag reflects ongoing disagreement about: (i) whether current patterns constitute "crisis" or normal science; (ii) replication-success definitions (significance-based; interval-based; combined-evidence; subjective); (iii) relative contributions of original-false-positives, replication-false-negatives, and true between-study heterogeneity; (iv) whether exact replication is philosophically coherent; (v) optimal reform direction (pre-registration-heavy; data-sharing-heavy; meta-science-heavy; incentive-reform-heavy). Related primes: #432 randomization (proper randomization as condition for reliable replication), #434 hypothesis_testing_null_vs_alternative (replication aggregates across single-study tests), #435 statistical_significance_p_value (p-value interpretation is compromised without replication expectation — contested_construct both primes), #437 statistical_power (under-powered originals produce winner's curse and non-replicable findings), #440 selection_bias (publication bias as selection-in-literature, primary replication-failure driver), #438 confounding (unmeasured confounders differing across contexts), #447 effect_size (replications measure effect-size persistence), #433 sampling_representativeness (replications in different populations test generalizability). Cross-DP-25 notes: reproducibility_replicability (this prime) is the validation framework for empirical inference; missing_data_mechanisms_mcar_mar_mnar identifies structural threats to complete-data assumptions; compression quantifies information content. Strong transfer targets: psychology and social-science replication reform, biomedicine and preclinical-research quality improvement, economics empirical-methodology consolidation, machine-learning computational reproducibility, genetics replication-cohort requirements, clinical-trial regulatory-replication standards, meta-science and science-of-science infrastructure. Pass B should develop archetypes for *pre-registration and registered reports*, *data-and-code sharing standards (FAIR principles, Dryad, OSF)*, *computational-reproducibility infrastructure (containers, notebooks, cloud)*, *many-labs and coordinated multi-site replication*, *meta-analytic replication synthesis*, *replication-study design (power, success criteria, context matching)*, *publication-bias correction in meta-analyses (funnel plots, trim-and-fill, p-curve, selection-model meta-analysis)*, and *reform-incentive infrastructure (replication journals, funding, tenure credit for replication)*.


Reputation

type: prime

slug: reputation

one_liner: An aggregated signal of past behaviour that shapes how others treat an agent.

status: draft

core_idea:
Reputation is the aggregated, publicly available record of an agent's past behavior that propagates beyond the parties to the original interactions and governs how third parties treat the agent in future ones, a structure the economics of repeated interaction first made precise when Klein and Leffler (1981) modeled how the present value of a future reputation premium disciplines current quality. <!-- FACT-D54-826 -->[^klein-leffler-1981] Its defining structure is information flow across two dimensions at once: across *time*, because conduct in the past constrains treatment in the future, and across *observers*, because behavior toward A becomes a signal that B, C, and D act upon. The agent's present incentives are therefore shaped not by the party currently in front of them but by the shadow their conduct casts on a wide and largely anonymous future audience. Reputation thereby converts a series of nominally one-shot encounters into something with the strategic texture of a repeated game, the mechanism Kreps and Wilson (1982) formalized when they showed that even a small probability of being a "tough" or honest type can sustain cooperative play in finitely repeated games. <!-- FACT-D54-827 -->[^kreps-wilson-1982]

What makes reputation a distinct structural primitive rather than a loose synonym for "good name" is that it is a *third-party-propagated stock*: it lives in the minds and records of observers who were never party to the original interaction, it accumulates and decays over time, and it can be consulted by a stranger who must decide how to treat the agent before any direct interaction has occurred. The prime names the general shape — observed conduct, transmitted across an audience, stored, and fed back as altered treatment — wherever that loop appears, whether the agents are firms, nations, species, web pages, or scientists.

structural_signature:
Reputation encodes a structural pattern: **observed conduct → third-party transmission → aggregated standing → altered future treatment → updated incentives**. It separates the *act* (a single observable behavior), the *channel* (the observers and records that carry it forward), the *stock* (the standing that accumulates in others' memories), and the *feedback* (the way that stock changes how the agent is treated and therefore what the agent does next). The loop closes when anticipated future treatment reaches back to discipline present conduct, the self-enforcing structure Nowak and Sigmund (2005) identified as the engine of indirect reciprocity. <!-- FACT-D54-828 -->[^nowak-sigmund-2005]

**Recurring features:**

- Aggregated public record of an agent's past behavior
- Standing that propagates beyond the original interacting parties
- Track record that strangers consult before any direct dealing
- Behavioral history compressed into a transferable token
- Shadow of the future cast onto present incentives
- Asset that is slow to build and fast to destroy
- Third-party information stock that sustains cooperation without enforcement

The structural insight is robust across substrates: a firm guarding brand equity, a nation cultivating a reputation for resolve, a cooperating organism accruing "good standing," a seller accumulating eBay feedback, and a web page gathering inbound links all exhibit the same loop in which past behavior, made observable to a future audience, becomes a stock that governs subsequent treatment, a convergence Resnick, Zeckhauser, Friedman, and Kuwabara (2000) documented when they showed that online reputation systems re-implement the indirect-reciprocity logic biologists had independently described. <!-- FACT-D54-829 -->[^resnick-2000]

what_it_is_not:
Reputation is **not** a private belief held by a single party. A buyer's individual confidence in a particular seller is a dyadic assessment; reputation is the *population-level* stock that exists even for observers who hold no belief yet because they have never interacted with the agent. The defining feature is that the standing is consultable by a stranger — it is information held in common, not a relationship held between two parties.

Reputation is **not** the same as actual quality or character. It is an *inference* about underlying type or behavior, and the inference can be wrong, stale, or manipulated. An agent can have a sterling reputation built on a record that no longer reflects current conduct (reputation lag), or a poor reputation generated by an unlucky run of observable outcomes that does not track true type. The prime names the propagated record and its effects, not the ground truth the record is supposed to summarize.

Reputation is **not** intrinsically good or deserved. A reputation for ruthlessness, for unreliability, or for cheating is still a reputation: the structure is symmetric with respect to valence. The loop sustains *cooperation* only when the channel reliably transmits the relevant behavior and the audience rewards good standing; the same loop can entrench a bad name, propagate slander, or lock an agent out of opportunity on the basis of a single damaging episode. Treating reputation as a synonym for "good reputation" smuggles in a value judgment the structure does not make.

Finally, reputation is **not** a single act of communication. People sometimes use "reputation" loosely for any attempt to manage how one is seen, but a one-off message or display is a signal; reputation is the *accumulated, third-party-propagated history* against which such signals are read, and which gives them their long-run stakes. The prime is the stock and its dynamics, not the momentary act of broadcasting.

broad_use:
**Economics & markets:** Brand reputation lets buyers infer unobservable quality before purchase; firms invest in costly-to-build, easy-to-destroy reputational capital precisely because the anticipated stream of repeat business disciplines present quality, the quality-assurance role Shapiro (1983) modeled by showing that a reputation premium can support prices above marginal cost as a bond against cheating. <!-- FACT-D54-830 -->[^shapiro-1983] Warranties, return policies, and certifications are instruments for transmitting or substituting for reputation.

**Evolutionary biology:** Indirect reciprocity sustains cooperation among non-kin when helping behavior earns the helper "good standing" that *third parties* later reward; the cooperation is carried not by direct payback from the recipient but by the propagated record itself, a mechanism whose stability under image-scoring dynamics Nowak and Sigmund (1998) demonstrated in their foundational simulations. <!-- FACT-D54-831 -->[^nowak-sigmund-1998]

**Computer science & distributed systems:** Reputation scores aggregate strangers' prior experiences so that parties with no shared history can transact — eBay feedback, web-of-trust key endorsement, and PageRank-style link authority all compute a standing from distributed observations, with PageRank treating an inbound hyperlink as a transferable endorsement of a page's authority, as Page, Brin, Motwani, and Winograd (1999) formalized. <!-- FACT-D54-832 -->[^page-1999] Adversarial concerns — Sybil attacks, collusive rating, whitewashing through identity change — are reputation-system attacks on the channel and the stock.

**International relations:** A state's reputation for resolve, or for honoring treaties and alliance commitments, shapes how adversaries and allies respond to its current moves; the credibility of deterrence and of promises rests on a propagated track record, the dynamic Schelling (1960) made central to his theory of commitment and bargaining. <!-- FACT-D54-833 -->[^schelling-1960]

**Academia & open source (non-obvious):** Citation counts, h-indices, and public commit histories function as portable reputation that allocates attention, hiring, funding, and trust across institutions the agent has never directly dealt with — the standing travels with the record rather than with the relationship, an attention-allocation effect Merton (1968) named the "Matthew effect" when he traced how prior eminence accrues disproportionate future recognition. <!-- FACT-D54-834 -->[^merton-1968]

clarity:
Naming reputation as a distinct prime separates three things that ordinary language runs together: the *standing* of an agent (a stock, held in many observers' minds and records), a single act of *signaling* (a flow, emitted to convey type), and the dyadic stance of *trust* (one party's willingness to be vulnerable to another). The distinction lets practitioners say something that is otherwise hard to express precisely: that an agent can be trusted by someone who has no direct experience of them at all, purely on transmitted reputation, and conversely that reputation can be damaged by behavior toward someone *other* than the party now deciding how to treat the agent. The harm to standing propagates through the audience, not through the injured party's own response. This separation also clarifies *whose* information reputation is: not the agent's (who may dispute it), not any single counterparty's (who holds only a fragment), but the aggregate carried by the observing population.

The clarity is practically load-bearing in mechanism design. Once one sees reputation as a stock fed by observable conduct and consulted by future strangers, the design questions become sharp: which behaviors are made observable, to whom, through what channel, with what memory and what decay, and how is the stock protected against forged or strategic inputs? Dellarocas (2003) argued that this reframing is exactly what lets online platforms engineer trust among strangers at scale — by deliberately constructing the observation-transmission-aggregation channel that physical communities once supplied through gossip. <!-- FACT-D54-835 -->[^dellarocas-2003]

manages_complexity:
Reputation compresses a long, dispersed behavioral history into a compact, transferable token, sparing each new counterparty the prohibitive cost of monitoring the agent from scratch or learning its type through costly trial. Instead of every newcomer re-running the experiment of dealing with an unknown agent, the population maintains a shared summary that any member can consult cheaply. This lets large populations of strangers coordinate *as if* they had a common memory, even though no individual holds the whole history. Milgrom, North, and Weingast (1990) showed how exactly this kind of institutionalized reputation tracking — the medieval law-merchant courts that recorded and transmitted traders' conduct — let merchants who would never meet again trade honestly across the fairs of Europe, substituting a reputation channel for the impossible task of bilateral enforcement among thousands of strangers. <!-- FACT-D54-836 -->[^milgrom-north-weingast-1990]

The complexity-management payoff is also a complexity *relocation*: the cost does not vanish but moves into the channel and the aggregator. Maintaining a trustworthy reputation system — verifying inputs, resisting manipulation, deciding how to weight old versus recent conduct — is itself demanding, which is why reputation institutions (rating agencies, credit bureaus, platform feedback systems, peer review) are durable, contested, and frequently captured.

abstract_reasoning:
Recognizing reputation as a prime enables folk-theorem reasoning in settings that look one-shot but are not. Because defection is observed and propagated to future counterparties, the threat of reputational loss can sustain cooperation without any external enforcer, as in the community-enforcement equilibria Kandori (1992) proved can support cooperation in large populations whose members rarely meet the same partner twice. <!-- FACT-D54-837 -->[^kandori-1992] The reasoning move is to ask, of any apparently isolated interaction: who is watching, what will they remember, and whom will they tell? If the answer is "a future audience that will adjust its treatment," the strategic structure is repeated-game-like even when the named parties never interact again.

The same lens exposes characteristic failure modes as structural rather than incidental: reputation *lag* (the stock reflects outdated behavior), *gaming* (agents invest in the observable proxy rather than the underlying conduct), *whitewashing* (an agent sheds a bad record by acquiring a fresh identity), and the *cliff-edge collapse* in which a long-accumulated record is destroyed by a single revelation. Tirole (1996) formalized the dynamics of such collapse and recovery, showing how an individual's reputation can be hostage to the past behavior of a group and why rehabilitation after a damaging episode is slow and uncertain. <!-- FACT-D54-838 -->[^tirole-1996]

knowledge_transfer: The biology of indirect reciprocity transfers directly to platform design: build a mechanism that makes past behavior observable to future counterparties, ensure the channel is hard to forge, and cooperation becomes self-enforcing without a central enforcer. This is not a loose analogy but a documented pipeline — online reputation-system designers explicitly drew on the indirect-reciprocity literature, and the structural correspondence (observed act, transmitted standing, third-party reward) is exact. Conversely, the economic insight that reputation is an asset with *hysteresis* — slow to build because it requires a long observable record, fast to lose because a single contrary signal can dominate the prior — transfers to diplomacy (a state's hard-won credibility for resolve evaporates with one backed-down threat) and to personal-credibility management. <!-- FACT-D54-840 -->[^klein-leffler-1981] The transferable reasoning is a checklist of channel properties: observability of conduct, fidelity of transmission, persistence and decay of the stock, and resistance to manipulation, which Bolton, Katok, and Ockenfels (2004) tested experimentally by comparing how different feedback-channel designs produce systematically different cooperation rates among the same population of traders. <!-- FACT-D54-839 -->[^bolton-katok-ockenfels-2004]

formal_abstract:
**Repeated-game / folk-theorem setting:** Consider a population of agents who are randomly matched in pairs to play a one-shot Prisoner's Dilemma, and who will almost never meet the same partner again. In a true one-shot game, defection is dominant and cooperation collapses. Now add a reputation channel: each agent carries an observable "image score" recording whether they cooperated or defected in recent interactions, visible to whomever they are matched with next. A strategy of "cooperate with partners in good standing, defect against those in bad standing" can now be an equilibrium: defecting earns a short-term gain but lowers the agent's score, inviting future partners to defect against them. The cooperation is carried entirely by the third-party-propagated record, not by any bilateral relationship. **Mapped back:** This is the prime's loop in its purest form — observed conduct becomes a transmitted standing that alters future treatment and so feeds back to discipline present conduct. The model isolates exactly the structure (a consultable stock summarizing past behavior across observers) that the prime names, stripped of any particular domain content; markets, alliances, and platforms are this same loop with different channels.

**PageRank as computed reputation:** PageRank treats the web as a population of pages whose "standing" is an aggregate of others' endorsements. A hyperlink from page A to page B is read as A vouching for B, and B's authority is the recursively computed sum of the authority flowing in through its inbound links. No page asserts its own importance; importance is a propagated stock constructed from the distributed behavior (linking) of the whole population. **Mapped back:** The pattern is reputation with hyperlinks as the observable conduct and the link graph as the transmission channel: a page's treatment by the ranking algorithm (where it appears in results) is governed by an aggregated record of how third parties have behaved toward it. Link-spam and link-farms are precisely channel attacks — attempts to forge the observable conduct that feeds the stock — which is why the structural failure modes of human reputation reappear, unchanged, in the computational substrate.

applied_industry:
**Online marketplace feedback (eBay-style):** A buyer must decide whether to send money to a seller they have never dealt with and will likely never deal with again. The marketplace supplies a reputation system: every past transaction leaves a rating, the ratings aggregate into a visible score and a written history, and the buyer consults this stock before transacting. Sellers, anticipating that future buyers will read the record, have an incentive to fulfill honestly even on a transaction whose own counterparty has no future leverage. The platform must defend the channel — preventing fake reviews, collusive rating rings, and "whitewashing" via fresh accounts — because the stock is only useful if its inputs are faithful. **Mapped back:** The platform has deliberately engineered the reputation loop that close-knit communities get for free through gossip: it manufactures observability (mandatory post-transaction ratings), transmission (a public profile), a persistent stock (the cumulative score), and feedback (buyers' altered willingness to transact). Every design lever — how long feedback persists, how much recent behavior is weighted, how identity is verified — is a choice about the channel and stock properties the prime identifies.

**Sovereign credibility and deterrence:** A state issues a deterrent threat — "if you cross this line, we will respond." Whether the threat works depends less on the state's present capabilities than on its reputation for actually carrying out such threats, built from a record of past behavior that adversaries and allies have observed and propagated. A state that has backed down before finds its current threats discounted; a state with a record of following through can deter with words alone. The reputation is an asset the state spends and replenishes: each crisis is partly a transaction in credibility, and a single conspicuous capitulation can collapse a standing built over decades. **Mapped back:** This is the prime operating with nations as agents, the international audience as the transmission channel, and credibility-for-resolve as the stock. The hysteresis property is vivid — credibility is slow to build (it requires a long observed record of follow-through) and fast to lose (one backed-down threat can dominate the prior) — which is exactly the asymmetry the economic treatment of reputation predicts.

title: Reputation rewards the observable proxy, not the underlying conduct it is supposed to summarize

body: Because the stock is built from what observers can see and record, agents face an incentive to invest in the observable signal rather than the substance behind it. A firm may polish its review profile while quality slips; a scholar may chase citable, fashionable work over careful but uncitable work; a seller may game the rating mechanism. The tighter and more consequential the reputation channel, the stronger the pull toward optimizing the measure rather than the thing measured. The structure that makes reputation useful — compression of conduct into a transmissible token — is the same structure that lets the token detach from the conduct.

title: Reputation is slow to build but fast to destroy, and this asymmetry is not symmetric in its costs

body: Building a stock requires a long observable record; a single damaging revelation can dominate the entire prior and collapse the standing overnight. This hysteresis disciplines behavior usefully (the threat of cliff-edge collapse keeps agents honest) but also makes reputation fragile and unforgiving: an agent who has behaved well for years can be ruined by one episode, sometimes one they did not commit, and rehabilitation is slow and uncertain even when the original record was overwhelmingly good.

title: The channel that makes cooperation self-enforcing is also an attack surface

body: Reputation sustains cooperation only when the transmission channel faithfully carries the relevant behavior. But the same channel can be forged (fake reviews, link farms), flooded (Sybil identities), captured (a rating agency paid by those it rates), or weaponized (coordinated slander). Strengthening the channel's reach and consequence raises the payoff to subverting it. Every reputation system therefore runs an arms race between the agents whose conduct it is meant to summarize and the manipulators who target the channel instead.

title: Reputation lets strangers coordinate, but only by importing the audience's biases

body: The promise of reputation is that a stranger can treat an agent appropriately without direct experience. But the stock the stranger consults is whatever the observing population chose to record and propagate, which carries that population's blind spots, prejudices, and uneven attention. The Matthew effect — prior standing attracting disproportionate further recognition — means reputation tends to amplify initial advantage rather than track current merit. Reputation can thus systematically misallocate trust and attention while appearing to be a neutral summary of behavior.

title: A bad reputation can be as self-reinforcing as a good one, trapping agents below a recovery threshold

body: The same feedback loop that rewards good standing punishes bad standing, and the punishment can foreclose the very interactions through which an agent might rebuild. An agent denied transactions, links, alliances, or citations because of a poor record gets no further chance to generate the contrary signals that would update the stock. Whitewashing — discarding the spoiled identity for a fresh one — becomes individually rational, which in turn forces reputation systems to penalize newcomers, raising the entry cost for genuinely new and well-intentioned agents.

title: Reputation presupposes a stable, trackable identity, yet identity is exactly what strategic agents can manipulate

body: The loop only closes if today's agent is reliably linked to the record of yesterday's conduct. When identity is cheap to change or hard to verify, the stock detaches from the agent: cheaters shed bad records, impostors borrow good ones, and the standing no longer governs the agent it was built from. Reputation systems are therefore forced into a costly identity-binding problem, and the strength of any reputation effect is capped by how firmly conduct can be pinned to a persistent identity.

prose: _No catalogued solution archetypes reference this prime yet._

notes:
Reputation operates at many scales and over many channels, and the prime is agnostic about which: the agent may be an individual, a firm, a nation, a species lineage, a web page, or an algorithm; the channel may be word-of-mouth gossip, a credit bureau, a platform's rating database, the citation graph, or the link structure of the web. What is invariant is the loop — observed conduct, transmitted across an audience, stored as a stock, fed back as altered treatment. When analyzing a concrete case it is usually most productive to name the four components explicitly: what conduct is observable, who carries it forward and how faithfully, where and how long the stock persists, and how the stock changes treatment.

A recurring confusion is between reputation and the *true* underlying property it is supposed to track (quality, type, reliability, resolve). The prime names the propagated record, which is an inference and can diverge arbitrarily from the truth. Much of the practical action in reputation systems is in managing this divergence — designing channels so that the stock tracks conduct closely, weighting recent behavior to limit lag, and binding records to persistent identities so manipulators cannot detach standing from agent.

Reputation also carries an implicit assumption that the observing population shares enough of a common interpretive frame to read conduct the same way. When audiences fragment — different communities recording different behaviors, or reading the same behavior with opposite valence — an agent can hold simultaneously good and bad reputations in different sub-populations, and the unitary "reputation" the prime names dissolves into several. This is increasingly visible in networked environments where reputation is computed differently by different platforms and communities.


Requisite Variety

one_liner: Match environmental complexity.

core_idea:
Requisite variety is the cybernetic constraint that matches a controller's response repertoire to its environment's disturbance repertoire such that:
(1) **only variety can absorb variety** (Ashby 1956)<!-- FACT-D26-037 -->[^ashby-1956] — formally, if a regulator $R$ must hold some essential variable $E$ within an acceptable set against disturbances $D$, then the variety of $R$ (its distinct internal states or response options) must be at least as large as the variety of $D$ divided by the variety $R$ can tolerate in $E$; in the strong form, $V(R) \geq V(D) / V(E)$ where $V(\cdot)$ denotes count or log-count of distinct states — a controller with fewer distinct responses than disturbance types is provably unable to maintain all essential variables within tolerance;
(2) requisite variety is fundamentally a **mathematical information-theoretic bound**, not a heuristic — Ashby derived it from set-theoretic considerations and it was later clarified in Shannon-information terms (Conant-Ashby 1970: "every good regulator of a system must be a model of that system"<!-- FACT-D26-038 -->[^conant-ashby-1970] — the regulator must encode a model of the environment's variety to match it); the theorem has the same status as Shannon's channel capacity: a fundamental limit that cannot be engineered around;
(3) the principle supplies **a diagnostic for under-powered control**<!-- FACT-D26-039 -->[^ashby-1958] — whenever a system persistently fails some subset of its environment, the requisite-variety frame prompts: *does the controller have enough distinct responses to match the disturbance set? If not, either reduce environmental variety (filter, constrain inputs) or increase controller variety (add response options) — no other strategy fixes the gap*; this sharpens analysis across security (defenders need variety matching attackers), healthcare (treatments need variety matching conditions), machine learning (models need capacity matching data variety), and organizational design (teams need skill variety matching problem variety);
(4) the concept appears across domains — **cybernetics and control theory** (Ashby's Law, Conant-Ashby good-regulator theorem, variety engineering in viable-system modeling)<!-- FACT-D26-040 -->[^wiener-1948], **organizational design** (Beer's Viable System Model uses requisite variety as core design criterion; Stafford Beer applied it to Chile's Cybersyn project<!-- FACT-D26-041 -->[^beer-project-cybersyn]; modern applications in enterprise architecture, agile team design, network-centric organizations), **machine learning and statistics** (model capacity must match data complexity — under-capacity models fail, over-capacity models overfit; hypothesis-class VC dimension bounds as requisite-variety analogs)<!-- FACT-D26-042 -->[^vapnik-chervonenkis], **security and defense** (diverse attack surfaces require diverse defenses; monoculture vulnerabilities; red-team / blue-team variety matching)<!-- FACT-D26-043 -->[^espejo-reyes-2011], **ecology** (biodiversity enables ecosystem response to perturbations; low-diversity ecosystems collapse under novel stressors)<!-- FACT-D26-044 -->[^holling-1973], **public policy** (one-size-fits-all policies fail heterogeneous populations; differentiated policies restore variety matching), **language and culture** (vocabulary and linguistic categories must match conceptual variety of experience; Sapir-Whorf weak form), **biology** (immune system's antibody variety matches pathogen variety; adaptive immunity is an explicit requisite-variety mechanism)<!-- FACT-D26-045 -->[^csete-doyle-2002] — all deploy the "variety-matches-variety" structural constraint.

structural_signature:
*the regulator-system variety-matching requirement*<!-- FACT-D26-031 -->[^ashby-1956]
*the law that "only variety can absorb variety" (Ashby)*<!-- FACT-D26-032 -->[^ashby-1956]
*the disturbance-set as system-perturbation source*<!-- FACT-D26-033 -->[^ashby-1958]
*the regulator-state-space size lower bound*<!-- FACT-D26-034 -->[^conant-ashby-1970]
*the model-as-good-as-system principle (Conant-Ashby)*<!-- FACT-D26-035 -->[^conant-ashby-1970]
*the variety-attenuation versus variety-amplification trade-off*<!-- FACT-D26-036 -->[^beer-1972]

A triple $(D, R, E)$ with $D$ = disturbance set (environmental variety), $R$ = regulator set (controller variety), $E$ = essential-variable tolerance set. A coupling $D \times R \to E$ describes how combined disturbance-and-response produces outcome states. Ashby's bound: for $R$ to maintain $E$ against all of $D$, the variety of $R$ (in bits or state count) must satisfy $V(R) \geq V(D) / V(E)$ where $V(E)$ is the tolerance-set variety (roughly, how much outcome-variety is acceptable). Equivalently, in information terms: the entropy reduction the regulator must achieve equals the disturbance entropy minus the tolerance entropy; the regulator must have channel capacity at least this reduction. The Conant-Ashby theorem additionally requires the regulator to contain a model of the environment isomorphic (in variety-generating structure) to the environment itself. Refinements include: distinction between variety and capacity (static repertoire vs response speed), variety attenuation by filtering, variety amplification by aggregation, and **variety engineering** (Beer) — the systematic design of variety matches across organizational layers.

what_it_is_not:
- Not merely **"more options is better"** — requisite variety specifies a *matched* variety constraint, not a "more is always better" heuristic. Excess controller variety is wasteful (more response options than disturbance types means idle capacity); matching is the design criterion.
- Not **complexity for its own sake** — the requisite-variety constraint applies only to variety relevant to disturbances and essential variables; irrelevant complexity does not satisfy requisite variety and may even degrade performance. The principle is about *targeted* variety matching.
- Not **redundancy** — redundancy is multiple copies of similar responses (increasing reliability but not variety); requisite variety is distinct, differentiated responses to different disturbances. Redundancy and variety are orthogonal design dimensions.
- Not a **heuristic or slogan** — Ashby's Law is a formal information-theoretic theorem with precise mathematical status. Many popularized versions reduce it to vague "match complexity with complexity" slogans; the theorem is sharper and makes concrete predictions (under-varietied controllers *provably fail* against high-variety disturbances).
- Not **model accuracy** exclusively — the Conant-Ashby good-regulator theorem shows the regulator must contain a model, but the model need not be accurate in every detail — it must be isomorphic in variety-generating structure. A cruder but structurally-adequate model can regulate; a detailed but structurally-wrong model cannot.

broad_use:
- **Cybernetics and systems (core domain)**: Ashby's 1956 *Introduction to Cybernetics* introduced the Law of Requisite Variety; Beer's Viable System Model (1970s) operationalizes it in organizational design; Conant-Ashby's good-regulator theorem (1970) deepens the information-theoretic framing. Variety engineering in Viable System Model analysis is a systematic methodology for diagnosing and correcting variety imbalances across organizational layers.
- **Organizational design and management**: Cross-functional team composition (teams need skill variety matching problem variety); enterprise architecture variety across business units; network-centric organizations (variety amplification through loose coupling); federated governance (variety matching across centralized-vs-decentralized decision structures). Beer's work on Chile's Cybersyn project (1971-73) was an explicit large-scale application of requisite variety to national-economic control (historically significant but curtailed by political events).
- **Machine learning and statistics**: Model capacity must match data complexity; under-fitted models (insufficient variety) cannot capture signal; over-parameterized models overfit. Vapnik-Chervonenkis (VC) dimension, Rademacher complexity, and modern analyses of neural-network capacity all quantify a variety-like property of hypothesis classes. Transfer learning and multi-task learning adjust the effective variety of a model to match the combined variety of target tasks.
- **Security and defense**: Diverse defenses against diverse attacks (defense in depth as variety engineering); monoculture vulnerabilities (when defenders share a single vulnerable configuration, single-variety attacks bypass all); red-team variety as a design principle (simulating adversary variety reveals defensive gaps); Byzantine-fault-tolerant systems amplify variety across replicas.
- **Ecology and resilience**: Biodiversity enables ecosystem response to perturbations (species richness correlates with response-function richness); low-diversity ecosystems collapse under novel stressors (monoculture crops vulnerable to new pathogens). Community ecology uses requisite-variety-like arguments to explain species-sensitivity analyses and assembly rules.
- **Public policy and healthcare**: One-size-fits-all policies fail heterogeneous populations (tax policies, education policies, healthcare delivery) — differentiated policies restore variety matching; personalized medicine attempts to match treatment variety to patient-phenotype variety.
- **Immunology**: Adaptive immune system generates antibody variety matching pathogen variety via V(D)J recombination and somatic hypermutation — one of biology's most explicit requisite-variety mechanisms. HIV's evolution of escape mutants is an ongoing arms race in variety matching between pathogen and immune system.
- **Language and cognition**: Vocabulary richness correlates with distinguishable concepts; linguistic relativity (Sapir-Whorf weak form) argues that categorical variety shapes cognitive variety; technical domains develop specialized vocabularies as requisite-variety responses to domain complexity.

clarity: Names the fundamental match-or-fail constraint between controller and environment that undergirds all of control, security, resilience, and learning. Without the requisite-variety frame, analysts may attribute control failures to "lack of effort" or "poor strategy" when the root cause is a structural variety mismatch that no strategy within the controller's repertoire can fix. With the frame, the analyst asks: *what is the disturbance variety? What is the controller variety? Is there a gap? If so, should we filter disturbances, or amplify controller variety?* This structural clarity distinguishes fixable problems (increase variety) from unfixable ones within the current architecture (the controller is too variety-poor to succeed against the disturbance set), enables principled diagnosis of persistent failures, and justifies investments in diversity (of skills, responses, configurations) as principled rather than decorative.

manages_complexity: Compresses control problems to a single variety-match question. Instead of enumerating every disturbance-response interaction, the requisite-variety frame asks: *does controller variety match disturbance variety?* This single question captures a necessary condition for success; if variety is insufficient, no detailed strategy can rescue the system. The frame also supports compositional variety analysis — large organizations decompose into subsystems each with their own disturbance-response coupling, and variety matching is analyzed layer by layer (Beer's recursive Viable System Model). This decomposition manages the complexity of large-system design by reducing it to per-layer variety checks. Variety filtering (reducing disturbance variety before it reaches the controller) and variety amplification (expanding controller repertoire through learning, collaboration, or tool use) are the two levers; engineers choose the balance based on cost and risk. The principle also supports impossibility results: a controller with variety $k$ cannot regulate a disturbance set with variety much greater than $k$, regardless of effort or strategy; this blocks wishful thinking about under-powered controllers.

abstract_reasoning: The requisite-variety abstraction asks: *what is the disturbance variety? What is the controller variety? Is there a mismatch, and if so, which direction? Should we filter the inputs (reducing variety to be handled) or amplify the controller (increasing response repertoire)? What are the variety-amplification mechanisms available — learning, tool use, collaboration, specialization — and what are their costs?* This transfers across cybernetic systems, organizational design, machine learning, security, ecology, and immunology. A mature analysis quantifies variety (state counts, entropy, VC dimension, species count) rather than gesturing vaguely at "complexity"; identifies the key essential variables and tolerance bands; and designs variety-matching interventions at the right layer. Immature analysis treats variety as a decoration ("diversity is good"), fails to quantify the mismatch, or attempts strategy fixes when the problem is structural (the controller simply lacks the responses to handle the disturbance set).

knowledge_transfer:
| Domain | Essential variable $E$ | Disturbance variety $V(D)$ | Controller variety $V(R)$ | Variety mechanism |
| --- | --- | --- | --- | --- |
| Thermostat | Room temperature | Weather variation | On/off control | Feedback loop |
| Enterprise org | Operational continuity | Market + internal shocks | Team skill set | Variety engineering |
| Machine learning | Prediction error | Training-data variety | Model capacity | Parameter count, depth |
| Security | Uncompromised system state | Attack surface diversity | Defense repertoire | Defense in depth |
| Ecosystem | Biomass stability | Environmental stressors | Species richness | Biodiversity |
| Public health | Population health | Disease variety | Treatment variety | Personalized medicine |
| Immune system | Pathogen-free host | Pathogen space | Antibody repertoire | V(D)J recombination |
| Governance | Citizen compliance | Population heterogeneity | Policy differentiation | Federalism, subsidiarity |
| Language | Communicable concepts | Experience diversity | Vocabulary size | Lexical growth |
| Chess/game AI | Win rate | Opponent strategy space | Move-evaluation variety | Search depth + evaluation |

Across rows, the variety-matches-variety pattern transfers with structural fidelity. Cross-domain transfer is a signature strength of requisite variety — an analyst trained in cybernetic variety engineering can apply the same analysis to machine-learning capacity, ecosystem-resilience design, immune-system vaccine strategy, or organizational-structure redesign.

formal_abstract:
**Ashby's Law applied to a regulator with a finite disturbance set**. Suppose a disturbance source produces events from set $D = \{d_1, d_2, \ldots, d_m\}$ with equal probability. A regulator $R$ has response set $R = \{r_1, r_2, \ldots, r_n\}$. The combined behavior is a function $f: D \times R \to E$ where $E$ is the essential-variable space; the regulator's policy is a function $\phi: D \to R$ (regulator sees disturbance, picks response). The regulator's goal is to keep $f(d, \phi(d))$ within an acceptable set $E_{\text{acc}} \subseteq E$ regardless of which $d \in D$ occurs. Question: how many distinct responses does $R$ need? Ashby's argument: if $|R| < |E_{\text{acc}}^{-1}|$ (the number of responses actually needed to map $D$ into $E_{\text{acc}}$), then no function $\phi$ can achieve the goal — some disturbance maps to an outcome outside $E_{\text{acc}}$. Formally, in terms of entropy: $H(E) \geq H(D) - H(R)$, so $H(R) \geq H(D) - H(E_{\text{acc}})$. The regulator's entropy (variety, in bits) must be at least the difference between disturbance entropy and tolerable outcome entropy. A toy case: if the disturbance set has 16 equally-likely events ($H(D) = 4$ bits) and the essential variable has 2-bit tolerance ($H(E_{\text{acc}}) = 1$), then the regulator needs at least $4 - 1 = 3$ bits of variety (8 distinct responses). A regulator with only 4 responses (2 bits) is provably inadequate. This quantitative form sharpens the "variety-matches-variety" principle into an information-theoretic lower bound and makes requisite variety a hard design constraint, not a soft heuristic. In cybernetic practice, variety is often counted as distinct *recognizable-and-handleable situations* rather than raw state space, but the formal structure remains.

**Mapped back:** Ashby's information-theoretic formalism directly constrains all control, regulation, and learning systems.

applied_industry:
An enterprise cybersecurity company builds its managed-detection-and-response platform around explicit requisite-variety reasoning for customer environments. The business problem: customers face an attack-variety that grows as adversaries automate, specialize, and diversify; a defense with too few response patterns will miss attacks that fall outside its repertoire, regardless of individual pattern sophistication. The team's design includes: (a) *threat taxonomy with variety mapping* — the platform maintains an up-to-date catalog of attack types (MITRE ATT&CK techniques, emerging exploit families, insider-threat patterns), quantifying the "variety" the defense must absorb; (b) *detection-rule diversity audits* — the platform regularly audits detection-rule coverage against the threat taxonomy, flagging "variety gaps" where rule coverage is thin; these audits are a direct requisite-variety analysis — rule count matters less than rule variety across threat types; (c) *red-team variety matching* — an internal red team simulates adversary variety, with explicit coverage targets for attack categories; blue-team detection rates across categories measure the variety match; (d) *response-playbook diversity* — incident-response playbooks cover distinct response patterns (isolate, monitor, rollback, forensic-collect, notify) matching the variety of attack outcomes; a monoculture response playbook would fail against diverse attack goals; (e) *tooling diversity as variety amplification* — the platform integrates multiple analysis engines (behavioral, signature-based, ML-based, heuristic) to amplify detection variety; single-engine reliance creates monoculture risk; (f) *variety metrics in dashboards* — customer dashboards include "coverage breadth" (how much of the threat taxonomy is addressed) and "response-breadth" (how many distinct response patterns are available) alongside traditional metrics; (g) *customer-environment variety adaptation* — the platform tunes detection and response to the customer's specific environment variety (cloud vs on-prem, user-count, regulatory regime); homogeneous environments need less variety than heterogeneous ones. The team's chief strategist describes the work as "Ashby's Law for cybersecurity": customers cannot buy their way out of variety mismatches by buying one powerful tool; they need distinct, differentiated capabilities matching the variety of what they face. This framing has commercial implications: the company's sales narrative emphasizes breadth of coverage rather than single-feature depth; product roadmaps prioritize filling variety gaps over enhancing already-covered areas; acquisitions target companies with complementary variety rather than overlapping strength. The practice is a direct, industrial-scale transfer of Ashby's Law into enterprise cybersecurity strategy.

**Mapped back:** Requisite-variety matching applies identically across cybersecurity threat-defense, organizational-design skill-matching, and regulatory policy differentiation.

prose:
_(Pass B — deferred)_

### Catalog cross-references (v1, batches 02–29)

Solution archetypes that reference this prime (per `solution_archetypes/_catalog/catalog_v1_batches_02_29.yaml`):

**As source prime (4):** `adaptive_capacity_building`, `local_rule_design`, `requisite_variety_matching`, `response_repertoire_expansion`

**As related prime (4):** `adaptive_reconfiguration`, `adaptive_response_recalibration`, `metasystem_integration`, `unity_variety_balancing`

notes: Systems-thinking/cybernetics-origin — introduced by W. Ross Ashby in *Introduction to Cybernetics* (1956) as the "Law of Requisite Variety." Stafford Beer generalized and operationalized it in the Viable System Model (1972) and applied it in Project Cybersyn (1971-73) in Chile. Conant-Ashby's good-regulator theorem (1970) strengthened the information-theoretic interpretation. Modern applications in organizational design (Beer's later work, Espejo), enterprise architecture, machine learning (capacity-control and regularization theory parallel requisite variety), and ecosystem resilience. Companion to #391 controllability (requisite variety is a necessary condition for controllability), #390 observability (dual notion — observability is requisite variety on the sensing side), #388 homeostasis (homeostasis uses requisite-variety-adequate regulators), #114 diversity_in_selection (biological requisite variety under evolution), and #375 complexity (requisite variety specifies the complexity match needed between controller and environment). Strong transfer targets: enterprise architecture and team design, cybersecurity defense portfolio design, machine-learning model-capacity sizing, ecosystem management and rewilding, public-policy differentiation for heterogeneous populations, and any design problem where "match the complexity of what you face" is a binding constraint. The principle is a foundational theorem of cybernetics and remains one of the most-cited ideas in systems thinking — its reach extends across every discipline where control, regulation, or matching is at stake.