‘Improving ratings’¶
Strathern, M. (1997). ‘Improving ratings’: audit in the British University system. European Review, 5(3), 305-321.
Cited by¶
58 citations across 58 artifacts.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Measurement
- In software and operations, metrics, telemetry, and dashboards make Goodhart's law the diagnostic: in Strathern's now-standard wording, when a measure becomes a target it ceases to be a good measure — a structural property of the measurement-feedback loop.
This sourceGives the generalized wording of Goodhart's law now in standard use: "When a measure becomes a target, it ceases to be a good measure."
- In software and operations, metrics, telemetry, and dashboards make Goodhart's law the diagnostic: in Strathern's now-standard wording, when a measure becomes a target it ceases to be a good measure — a structural property of the measurement-feedback loop.
Domain-specific¶
- Feature Factory
- the broader structural pattern — a proxy measure displacing the target it was meant to track — is Goodhart's Law applied to the product operating loop
This sourceThe source of the generalisation 'when a measure becomes a target, it ceases to be a good measure'.
- the broader structural pattern — a proxy measure displacing the target it was meant to track — is Goodhart's Law applied to the product operating loop
- Gold-Standard Erosion
- Reading the drop as the model degrading and reaching for retraining is the specifically wrong fix the construct exists to prevent. Not Goodhart's law. Under Goodhart, a measure becomes a target and decays because it is optimized against; the corruption is endogenous to the apparatus using the measure.
This sourceGives the canonical formulation that when a measure becomes a target it ceases to be a good measure, the corruption arising from optimisation by the system being measured.
- Reading the drop as the model degrading and reaching for retraining is the specifically wrong fix the construct exists to prevent. Not Goodhart's law. Under Goodhart, a measure becomes a target and decays because it is optimized against; the corruption is endogenous to the apparatus using the measure.
Mechanisms¶
- Abuse-Case Replay Harness
- Its central failure mode is overfitting to the replay set: a team can tune a mitigation until it aces the fixed corpus while remaining blind to the variant that is not in it, and a green harness run then reads as far more assurance than it is — the metric has become the target rather than a proxy for it.
This sourceStates the Goodhart-style failure in which an evaluative measure degrades when it is turned into the target.
- Its central failure mode is overfitting to the replay set: a team can tune a mitigation until it aces the fixed corpus while remaining blind to the variant that is not in it, and a green harness run then reads as far more assurance than it is — the metric has become the target rather than a proxy for it.
- Adoption Analytics and Field Review
- Its failure mode is trusting the very metric it exists to interrogate — Goodhart's law warns that once a usage number becomes the target, people optimize the number rather than the work, and the dashboard measures compliance theater instead of adoption.
This sourceExplains how audit targets redirect organizational effort toward improving the measured rating rather than the underlying work.
- Its failure mode is trusting the very metric it exists to interrogate — Goodhart's law warns that once a usage number becomes the target, people optimize the number rather than the work, and the dashboard measures compliance theater instead of adoption.
- Adoption Dashboard
- Its central failure mode is metric theater: the moment a tracked number becomes the goal
This sourceStates Goodhart's law that a measure ceases to be reliable when it becomes a target, as target-driven assessment reshapes the activity being measured.
- Its central failure mode is metric theater: the moment a tracked number becomes the goal
- Aggregation Rules
- Once a composite becomes a target, it also invites gaming — people optimize the score rather than the thing it was meant to summarize.
This sourceStates that when a measure becomes a target it ceases to be a good measure, as target pressure shifts attention toward improving the measured rating.
- Once a composite becomes a target, it also invites gaming — people optimize the score rather than the thing it was meant to summarize.
- Agile Portfolio Governance
- Its failure mode arrives through the numbers: when a status metric or velocity figure becomes the thing the review rewards, teams optimize the metric rather than the outcome, and the signal stops meaning what it did — a measure that becomes a target ceases to be a good measure.
This sourceExplains that turning a measure into a target redirects effort toward the rating and erodes the measure's value as a signal.
- Its failure mode arrives through the numbers: when a status metric or velocity figure becomes the thing the review rewards, teams optimize the metric rather than the outcome, and the signal stops meaning what it did — a measure that becomes a target ceases to be a good measure.
- Algorithmic Ranking Audit
- And an audit tied to a fixed metric invites Goodhart's law
This sourceAnalyzes how audit culture conflates measures with targets, allowing assessment practices to distort the activity being measured.
- And an audit tied to a fixed metric invites Goodhart's law
- Automated Conformance Check
- And because pass rate is so visible, it invites optimizing to the check rather than to the underlying quality
This sourceStates Goodhart's law: when a measure becomes a target, it ceases to be a good measure.
- And because pass rate is so visible, it invites optimizing to the check rather than to the underlying quality
- Base Health Dashboard
- The sharpest misuse appears once a tracked metric becomes a target and is gamed rather than genuinely met — closing tickets fast to hit a backlog number while the underlying issues fester, an instance of what is known as Goodhart's Law.
This sourceStates Goodhart’s Law as the loss of a measure’s usefulness once it becomes a target.
- The sharpest misuse appears once a tracked metric becomes a target and is gamed rather than genuinely met — closing tickets fast to hit a backlog number while the underlying issues fester, an instance of what is known as Goodhart's Law.
- Behavior Data Dashboard
- What is easy to count crowds out what matters, and once a displayed number becomes the target people manage to the number rather than the purpose — the pattern Goodhart's law names.
This sourceStates Goodhart’s law as the loss of a measure’s value once it becomes a target and shows how measurable outputs can crowd out less visible activities.
- What is easy to count crowds out what matters, and once a displayed number becomes the target people manage to the number rather than the purpose — the pattern Goodhart's law names.
- Boundary Permeability Scorecard
- Its failure mode is that scores invite false precision, and a score that becomes a target stops measuring reality — an instance of Goodhart's law.
This sourceStates Goodhart's law: when a measure becomes a target, it ceases to function as a good measure.
- Its failure mode is that scores invite false precision, and a score that becomes a target stops measuring reality — an instance of Goodhart's law.
- Bycatch Rate Dashboard
- The classic misuse is building or slicing it after the fact to reassure — choosing the segmentation and the harm weights that make bycatch look negligible, which is exactly the failure mode Goodhart's law warns of: once the on-screen number becomes the thing being managed, it stops measuring the thing that matters.
This sourceExplains that managing to a measure as a target degrades its value as a measure of the underlying performance.
- The classic misuse is building or slicing it after the fact to reassure — choosing the segmentation and the harm weights that make bycatch look negligible, which is exactly the failure mode Goodhart's law warns of: once the on-screen number becomes the thing being managed, it stops measuring the thing that matters.
- Cohesion Pulse Survey
- Once a cohesion score becomes a target that managers are judged on, it stops being a good measure of cohesion
This sourceExplains that judging managers on a cohesion score turns it into a target and degrades its value as a measure.
- Once a cohesion score becomes a target that managers are judged on, it stops being a good measure of cohesion
- Competency Matrix Update
- And once the cells become performance targets, people optimize the record rather than the skill — Goodhart's law.
This sourceStates Goodhart's law: when a measure becomes a target, it ceases to be a good measure.
- And once the cells become performance targets, people optimize the record rather than the skill — Goodhart's law.
- Contest or Autograder Harness
- A wrong solution that happens to satisfy every held-out case is accepted, and entrants who can see or infer the tests will tune to them — Goodhart's law in action
This sourceStates Goodhart’s law: turning a measure into a target degrades its ability to represent the outcome it was meant to assess.
- A wrong solution that happens to satisfy every held-out case is accepted, and entrants who can see or infer the tests will tune to them — Goodhart's law in action
- Contest Rulebook
- Its central failure is that a rulebook is only as good as its scoring metric — rivals optimize exactly what is written, so a proxy measure gets gamed the moment it becomes the target.
This sourceExplains that a proxy can be gamed once it becomes a target.
- Its central failure is that a rulebook is only as good as its scoring metric — rivals optimize exactly what is written, so a proxy measure gets gamed the moment it becomes the target.
- Contribution Review Cadence
- Once contributors know exactly which numbers the quarterly review watches, the numbers start to be managed rather than the good — Goodhart's law in miniature, where a measure adopted as a target stops measuring what it used to.
This sourceStates the Goodhart-style result that a measure ceases to measure well once it is made a target.
- Once contributors know exactly which numbers the quarterly review watches, the numbers start to be managed rather than the good — Goodhart's law in miniature, where a measure adopted as a target stops measuring what it used to.
- Debt Aging Dashboard
- Its failure mode is that a dashboard measures what is easy to instrument, and a rolled-up health index invites Goodhart's law: once a subsystem score becomes a target teams are judged by, they optimize the score rather than the health it was meant to proxy, and the number stays green while the reality rots.
This sourceExplains that when a performance measure becomes a target it can become a poorer measure and reshape the activity being assessed.
- Its failure mode is that a dashboard measures what is easy to instrument, and a rolled-up health index invites Goodhart's law: once a subsystem score becomes a target teams are judged by, they optimize the score rather than the health it was meant to proxy, and the number stays green while the reality rots.
- Decision Method Triage Matrix
- Its characteristic failure is bin-gaming: once actors learn which cell routes to the cheap lane, they shade the inputs to land there — a live instance of Goodhart's law,
This sourceStates Goodhart’s law: when a measure becomes a target, it ceases to be a good measure.
- Its characteristic failure is bin-gaming: once actors learn which cell routes to the cheap lane, they shade the inputs to land there — a live instance of Goodhart's law,
- Discretion Audit Dashboard
- Its sharpest failure is Goodhart's law
This sourceStates Goodhart's law and shows how making a measure a target shifts behavior toward the measured number.
- Its sharpest failure is Goodhart's law
- Drain Dashboard
- It misleads when a single count becomes the thing everyone optimises
This sourceStates that a measure can cease to be a good measure when it becomes a target.
- It misleads when a single count becomes the thing everyone optimises
- Emergent Behavior Dashboard
- Its central failure is Goodhart's law: once a displayed metric becomes the target teams are judged on, it stops measuring what it did and starts being gamed or gardened.
This sourceStates that a measure ceases to be a good measure when it becomes a target.
- Its central failure is Goodhart's law: once a displayed metric becomes the target teams are judged on, it stops measuring what it did and starts being gamed or gardened.
- Express Lane
- The first is criterion gaming: once "15 items" becomes a target rather than a description, people optimize against it, and the lane fills with almost-qualifying baskets — a plain case of Goodhart's law.
This sourceStates Goodhart's law: when a measure becomes a target, it ceases to be a good measure.
- The first is criterion gaming: once "15 items" becomes a target rather than a description, people optimize against it, and the lane fills with almost-qualifying baskets — a plain case of Goodhart's law.
- Hard-Negative Data Augmentation
- The sharpest misuse is Goodharting the probe: augmenting specifically to beat one reliance test makes that test go green while the underlying reliance persists on everything it does not measure.
This sourceExplains that when an evaluative measure becomes a target, actors adapt to it and the measure can cease to represent the underlying activity.
- The sharpest misuse is Goodharting the probe: augmenting specifically to beat one reliance test makes that test go green while the underlying reliance persists on everything it does not measure.
- Harm Reduction Dashboard
- Its failure mode is metric fixation: the dashboard becomes the goal, and the pathway gets optimized for green cells rather than for real relief — an instance of Goodhart's law, where a measure that becomes a target stops measuring what it did.
This sourceStates Goodhart's law: when a measure becomes a target, it ceases to be a good measure.
- Its failure mode is metric fixation: the dashboard becomes the goal, and the pathway gets optimized for green cells rather than for real relief — an instance of Goodhart's law, where a measure that becomes a target stops measuring what it did.
- Institutional Health Dashboard
- Its central failure mode is Goodhart's law: once an indicator becomes a target, people optimize the number rather than the health it stood for, and the board can glow green while the institution rots underneath
This sourceStates Goodhart’s law as the loss of a measure’s usefulness once it is made a target.
- Its central failure mode is Goodhart's law: once an indicator becomes a target, people optimize the number rather than the health it stood for, and the board can glow green while the institution rots underneath
- Internal Service Agreement
- Once a turnaround number becomes the target the specialist is judged on, effort migrates to hitting the metric — closing tickets fast, gaming the intake categories — rather than to the underlying help, an instance of Goodhart's law.
This sourceStates that a measure ceases to be a good measure once it becomes a target, identifying the dynamic as Goodhart's law.
- Once a turnaround number becomes the target the specialist is judged on, effort migrates to hitting the metric — closing tickets fast, gaming the intake categories — rather than to the underlying help, an instance of Goodhart's law.
- KPI Governance Review
- Its strength is catching metric fixation before it hardens: it is the direct organizational answer to Goodhart's law — once a measure becomes a target, it ceases to be a good measure
This sourceStates that a measure ceases to be a good measure when it becomes a target.
- Its strength is catching metric fixation before it hardens: it is the direct organizational answer to Goodhart's law — once a measure becomes a target, it ceases to be a good measure
- Maximum Wait Guarantee
- An impossible guarantee becomes a trust-damaging lie, and once a hard target exists it invites gaming: clocks get reset, cases get split, or items get superficially "touched" to dodge a breach — the target is hit while the point is missed.
This sourceAudit targets can distort the activity they measure, allowing measured success while the underlying purpose is harmed.
- An impossible guarantee becomes a trust-damaging lie, and once a hard target exists it invites gaming: clocks get reset, cases get split, or items get superficially "touched" to dodge a breach — the target is hit while the point is missed.
- Membership Threshold Dashboard
- And once a displayed metric becomes the number everyone manages to, it invites gaming
This sourceExplains that when a measure becomes a target, actors' responses can undermine its validity as a measure.
- And once a displayed metric becomes the number everyone manages to, it invites gaming
- Mobile-Defect Progress Dashboard
- The classic misuse follows directly: the progress number becomes the target and gets gamed, so operators push "percent done" upward while quietly deferring the hardest units — 95% complete concealing the worst 5%.
This sourceExplains why a measure loses discriminatory value once it becomes a target for improvement.
- The classic misuse follows directly: the progress number becomes the target and gets gamed, so operators push "percent done" upward while quietly deferring the hardest units — 95% complete concealing the worst 5%.
- Network Capacity Dashboard
- Teams start managing to the dashboard — optimizing the green tiles rather than the flow, the moment a displayed metric becomes the target.
This sourceStates Goodhart’s law: when a measure becomes a target, it ceases to be a good measure.
- Teams start managing to the dashboard — optimizing the green tiles rather than the flow, the moment a displayed metric becomes the target.
- Objective Drift Dashboard
- Its failure mode is the one that stalks every dashboard: Goodhart's Law
This sourceStates Goodhart’s law and shows how targeting a measure redirects conduct toward improving the rating rather than the underlying activity.
- Its failure mode is the one that stalks every dashboard: Goodhart's Law
- Organizational Level Mapping
- It is the clearest lens on Goodhart's Law: once the velocity metric became the target, it stopped measuring healthy speed and started producing corner-cutting that the metric could not see.
This sourceGoodhart's Law warns that a measure can cease to be a good measure once it becomes a target.
- It is the clearest lens on Goodhart's Law: once the velocity metric became the target, it stopped measuring healthy speed and started producing corner-cutting that the metric could not see.
- Outcome Monitoring Review
- The classic trap is Goodhart's law: once a monitored number becomes the target
This sourceStates Goodhart’s law as the loss of a measure’s discriminating value once that measure becomes a target.
- The classic trap is Goodhart's law: once a monitored number becomes the target
- Oversight Dashboard
- Worse, once a displayed metric becomes the thing people are judged on, they optimize the number rather than the reality behind it, and the signal quietly decouples from the truth.
This sourceShows how judging institutions by a displayed measure creates pressure to optimize the measure until it ceases to represent the underlying performance.
- Worse, once a displayed metric becomes the thing people are judged on, they optimize the number rather than the reality behind it, and the signal quietly decouples from the truth.
- Pattern Monitoring Dashboard
- It misleads because you only see what you instrument, and once a metric is watched it tends to get gamed
This sourceExplains that a measure can cease to be a good measure once it becomes a target and is manipulated for improvement.
- It misleads because you only see what you instrument, and once a metric is watched it tends to get gamed
- Policy Refinement Cycle
- Its sharpest failure is Goodhart's law: once a compliance number becomes the thing being managed, actors optimize the number rather than the outcome, and the rule "converges" on a metric that no longer means what it did.
This sourceStates Goodhart’s law and shows how turning an audit measure into a target degrades its ability to represent the performance it was meant to assess.
- Its sharpest failure is Goodhart's law: once a compliance number becomes the thing being managed, actors optimize the number rather than the outcome, and the rule "converges" on a metric that no longer means what it did.
- Proxy Drift and Goodhart Audit
- Its strength is catching slow validity decay that any one-time validation misses entirely — the construct–proxy link that was fine on day one and hollow by month twelve, precisely because the proxy became a target.
This sourceStates that a measure can cease to be a good measure when it becomes a target.
- Its strength is catching slow validity decay that any one-time validation misses entirely — the construct–proxy link that was fine on day one and hollow by month twelve, precisely because the proxy became a target.
- Proxy Drift Dashboard
- It is the operational answer to Goodhart's law, the rule that a measure adopted as a target ceases to be a good measure once people optimize it.
This sourceStates that a measure ceases to be a good measure when it becomes a target.
- It is the operational answer to Goodhart's law, the rule that a measure adopted as a target ceases to be a good measure once people optimize it.
- Proxy Validity Audit
- This is the mechanism that operationalises the warning behind Goodhart's law, that a measure adopted as a target tends to stop being a good measure.
This sourceA measure can cease to be a good measure once it becomes a target.
- This is the mechanism that operationalises the warning behind Goodhart's law, that a measure adopted as a target tends to stop being a good measure.
- Ranked Leaderboard With Audit
- Its strength is that a shared, visible metric aligns effort and lets everyone see the frontier, while the audit is what stops the board from simply rewarding whoever games hardest — it keeps the measure honest against Goodhart's law.
This sourceDocuments it keeps the measure honest against Goodhart's law.
- Its strength is that a shared, visible metric aligns effort and lets everyone see the frontier, while the audit is what stops the board from simply rewarding whoever games hardest — it keeps the measure honest against Goodhart's law.
- Reservoir Balance Dashboard
- The classic misuse is curating the display until it always reads green: once the shown level becomes the number people manage, they optimise the indicator instead of the reservoir, and the monitor stops reflecting reality.
This sourceExplains that when an indicator becomes an audit target, actors optimize the measure and its relation to the underlying activity deteriorates.
- The classic misuse is curating the display until it always reads green: once the shown level becomes the number people manage, they optimise the indicator instead of the reservoir, and the monitor stops reflecting reality.
- Residual-Risk Monitoring Dashboard
- Its failure modes are the classic ones for any instrument: a green board breeds false confidence, measuring the measurable while the real risk sits off-dashboard, and indicators invite gaming — a side manages the numbers rather than the risk, so the board stays calm while danger grows.
This sourceArgues that audit indicators become unreliable when target pressure causes people to manage the measure itself.
- Its failure modes are the classic ones for any instrument: a green board breeds false confidence, measuring the measurable while the real risk sits off-dashboard, and indicators invite gaming — a side manages the numbers rather than the risk, so the board stays calm while danger grows.
- Risk Decay Dashboard
- Its danger is Goodhart's law
This sourceStates Goodhart's law: when a measure becomes a target, it ceases to be a good measure.
- Its danger is Goodhart's law
- Safety-Gain Offset Dashboard
- And once a net-safety number becomes the goal, teams optimize the number: drop an inconvenient substitution channel, and the net looks positive by construction.
This sourceDocuments once a net-safety number becomes the goal, teams optimize the number.
- And once a net-safety number becomes the goal, teams optimize the number: drop an inconvenient substitution channel, and the net looks positive by construction.
- Service Desk
- This is Goodhart's law
This sourceStates that when a measure becomes a target, it ceases to be a good measure.
- This is Goodhart's law
- Service-Level or Internal Service Agreement
- Once a service level becomes a target, people optimize the number rather than the underlying outcome — closing tickets fast to hit the turnaround clock while the consumer's actual problem goes unsolved — the familiar way a measure that becomes a target ceases to be a good measure.
This sourceStates that when a measure becomes a target it ceases to be a good measure because target pressure changes what the metric records.
- Once a service level becomes a target, people optimize the number rather than the underlying outcome — closing tickets fast to hit the turnaround clock while the consumer's actual problem goes unsolved — the familiar way a measure that becomes a target ceases to be a good measure.
- Source Control Linkage
- That is Goodhart's law operating on a trace convention
This sourceStates that when a measure becomes a target, it ceases to be a good measure.
- That is Goodhart's law operating on a trace convention
- Substance-over-Form Audit
- And once compliance is measured, the measure becomes a target
This sourceExplains that audit measures become targets and that target-driven audit reshapes conduct around auditable forms.
- And once compliance is measured, the measure becomes a target
- Test Coverage Audit
- Its failure mode is that coverage is a proxy: 100% line coverage with weak assertions verifies nothing, and a coverage number invites Goodhart's law — once the percentage becomes the target, teams write tests that lift it without testing anything.
This sourceStates that a measure ceases to function well as a measure when it is made a target.
- Its failure mode is that coverage is a proxy: 100% line coverage with weak assertions verifies nothing, and a coverage number invites Goodhart's law — once the percentage becomes the target, teams write tests that lift it without testing anything.
- Weighted Scoring Model
- Its failure mode is proxy optimization — Goodhart's law
This sourceExplains that a proxy can cease to track its objective once it becomes a target for optimization.
- Its failure mode is proxy optimization — Goodhart's law
- Work-Before-Metawork Ratio Dashboard
- Its failure mode is Goodhart's Law.
This sourceStates that when a measure becomes a target, it ceases to be a good measure.
- Its failure mode is Goodhart's Law.
- Workload Heatmap
- Its characteristic failure is that a comparative color grid is almost engineered to be misused as a leaderboard, and the moment a heatmap becomes a target the measured cells stop tracking real contribution — a textbook case of Goodhart's law
This sourceExplains the Goodhart-type failure in which a measure subjected to target pressure ceases to track the activity it was meant to assess.
- Its characteristic failure is that a comparative color grid is almost engineered to be misused as a leaderboard, and the moment a heatmap becomes a target the measured cells stop tracking real contribution — a textbook case of Goodhart's law
- Zone Health Dashboard
- Its signature failure is Goodhart's law — once a displayed metric becomes the target, teams optimize the indicator and it decouples from the health it was meant to signal, so a green dashboard can mask a sick zone.
This sourceStates Goodhart’s law: when a measure becomes a target, it ceases to be a good measure because target pressure degrades its discriminating function.
- Its signature failure is Goodhart's law — once a displayed metric becomes the target, teams optimize the indicator and it decouples from the health it was meant to signal, so a green dashboard can mask a sick zone.
Verification¶
Does it exist? Not checked yet. This work's DOI is recorded above but has not been resolved against an external catalogue, so nothing here confirms the work exists.
Does it back the claim? Not recorded. Neither this nor any other of the 58 citations of this work carries a recorded support check.
Was it audited? Yes. A second, independent pass read the citation against the article text and recorded a verdict.
Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.
See how references were verified.
Links previously used in the corpus¶
Before the registry existed this work was also linked 6 other ways.
- https://doi.org/10.1017/S1062798700002660 ×20
- https://doi.org/10.1002/%28SICI%291234-981X%28199707%295%3A3%3C305%3A%3AAID-EURO184%3E3.0.CO%3B2-4 ×12
- https://doi.org/10.1002/(sici)1234-981x(199707)5:3<305::aid-euro184>3.0.co;2-4 ×2
- https://www.cambridge.org/core/journals/european-review/article/improving-ratings-audit-in-the-british-university-system/FC2EE640C0C44E3DB87C29FB666E9AAB ×2
- https://www.cambridge.org/core/product/identifier/S1062798700002660/type/journal_article ×2
- https://www.cambridge.org/core/journals/european-review/article/abs/improving-ratings-audit-in-the-british-university-system/FC2EE640C0C44E3DB87C29FB666E9AAB ×1
Registry ID ref:80af5b95ad0e · see in the full table