Categorizing Variants of Goodhart's Law¶
Manheim, D., & Garrabrant, S. (2019). Categorizing Variants of Goodhart's Law. arXiv.
Cited by¶
6 citations across 6 artifacts.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Agency
- The same structure governs platform content-moderation (creators re-plan around any published rule), sanctions regimes (targeted states develop counter-strategies and grey-market routes), and clinical-quality metrics (clinicians avoid high-risk patients to protect their scores) — in each, treating an agent as an object guarantees the surprise.
Supported in partVerified against the work's full text
Supplies the general adversarial-Goodhart mechanism — an agent re-plans against a regulator's imposed metric — but not the content-moderation, sanctions, or clinical-metrics cases, nor a guarantee.
“There are clearly further dynamics worth exploring, but this case s erves to introduce the issues involved in adversarial conflict over metrics without incentives.”
- The same structure governs platform content-moderation (creators re-plan around any published rule), sanctions regimes (targeted states develop counter-strategies and grey-market routes), and clinical-quality metrics (clinicians avoid high-risk patients to protect their scores) — in each, treating an agent as an object guarantees the surprise.
- Goodhart's Law
- The Goodhart diagnostic identifies all of these as exploitation of the construct-proxy wedge under control-loop pressure, and the same structural moves appear in the Soviet nail-factory legend, account-opening quotas, GDP-target manipulation, and language-model truthfulness proxies optimized into confident-sounding plausibility.
This sourceTaxonomizes Goodhart effects (regressional, extremal, causal, adversarial), unifying Campbell's law, the McNamara fallacy, surrogation, reward hacking, and p-hacking as one mechanism in different substrates.
Supported in partVerified against the publisher's abstract
Manheim & Garrabrant supply the Goodhart mechanism the sentence rests on — optimization pressure on a proxy — but not the article's enumerated exemplars or its unifying 'construct-proxy wedge' diagnostic.
“The importance of Goodhart effects depends on the amount of power directed towards optimizing the proxy, and so the increased optimization power offered by artificial intelligence makes it especially critical for that field.”
- The Goodhart diagnostic identifies all of these as exploitation of the construct-proxy wedge under control-loop pressure, and the same structural moves appear in the Soviet nail-factory legend, account-opening quotas, GDP-target manipulation, and language-model truthfulness proxies optimized into confident-sounding plausibility.
- Side Effect
- Public health and machine learning: campaigns and deployed models aimed at one outcome shift others (weight gain, vaping uptake; user behavior, content ecosystems), the reward-hacking and Goodhart literatures being a class of side effects.
This sourceTaxonomy of Goodhart and reward-hacking failure modes where optimizing a declared metric degrades the underlying substrate.
Supported in partVerified against the work's full text
Backs only the conceptual half: a taxonomy of Goodhart/reward-hacking failure where metric optimization degrades the substrate; it supplies no evidence for the named public-health or content-ecosystem cases.
“Selection on the basis of the approximated m etric moves towards regions where the higher-order terms are more importan t, so that use of the machine learning system creates a Goodhart effect.”
- Public health and machine learning: campaigns and deployed models aimed at one outcome shift others (weight gain, vaping uptake; user behavior, content ecosystems), the reward-hacking and Goodhart literatures being a class of side effects.
Domain-specific¶
- Vanity-Metric Addiction
- The first is the first layer: a measured proxy diverging from the outcome it was meant to track, especially under optimization pressure — proxy-target divergence, the Goodhart's-Law family — which recurs as genuine co-instances across domains that share no product-analytics machinery: Campbell's law and teaching-to-the-test in education, the Lucas critique in macroeconomics, target-fixation in policy, and reward hacking in reinforcement learning
This sourceA taxonomy of Goodhart's-law failure modes that treats Campbell's law and the Lucas critique as related formulations and applies them to policy, education and machine learning.
Supported in partVerified against the source
- The first is the first layer: a measured proxy diverging from the outcome it was meant to track, especially under optimization pressure — proxy-target divergence, the Goodhart's-Law family — which recurs as genuine co-instances across domains that share no product-analytics machinery: Campbell's law and teaching-to-the-test in education, the Lucas critique in macroeconomics, target-fixation in policy, and reward hacking in reinforcement learning
Mechanisms¶
- Organizational Health by Unit Monitoring
- Its failure mode is Goodhart's law
This sourceExplains that strong optimization pressure on a proxy can improve the measured proxy while degrading its relationship to the underlying objective.
- Its failure mode is Goodhart's law
- Process Metric
- Its failure begins the moment the metric becomes a target: Goodhart's Law warns that a measure optimized as a goal stops measuring what it did, as teams learn to move the number without moving the underlying state.
This sourceExplains Goodhart-type failures in which optimization against a metric makes further metric optimization ineffective or harmful.
- Its failure begins the moment the metric becomes a target: Goodhart's Law warns that a measure optimized as a goal stops measuring what it did, as teams learn to move the number without moving the underlying state.
Verification¶
Does it exist? Confirmed. This work's DOI resolves to a registered record, which fixes its identity. That is all it fixes.
Does it back the claim? Read against the text for 4 of 6 citations: 4 supported in part. Each verdict is shown under its citation below, with what in the work backs the sentence.
Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.
See how references were verified.
Links previously used in the corpus¶
Before the registry existed this work was also linked 1 other way.
Registry ID ref:22e44978165f · see in the full table