Anti-Gaming Scoring Rule¶
Scoring rule — instantiates Incentive-Compatible Rule Design
Scores behavior so the top score is earned by producing the real outcome, not by manipulating the measured proxy, and re-tunes as gaming emerges.
An Anti-Gaming Scoring Rule is a measurement — a score, index, or ranking — deliberately engineered so that the cheapest way to raise it is to do the thing the system actually wants, not to exploit the metric. Every score is a proxy for some real outcome, and any gap between proxy and outcome is a loophole a participant can profit from without helping. The distinctive move here is to close that gap by design: choose, compose, and risk-adjust the measure so that superficial compliance, cherry-picking, and proxy-optimization stop paying, then keep watching the deployed metric for the new games participants invent. It is only a measure — it says nothing about what reward or penalty rides on the score, which is what separates it from its enforcement-side siblings.
Example¶
A regional health authority wants to publish surgeon-level results so patients can choose well and quality improves. The naive version scores each cardiac surgeon on raw 30-day mortality. Within a year the score is worse than useless: the safest way to a good number is not better surgery but patient selection — surgeons quietly decline the sickest, highest-risk cases, which get shunted elsewhere or refused care. The metric improved while the real outcome (good care for everyone who needs it) deteriorated.
The redesign turns it into an anti-gaming score. Each case is risk-adjusted for the patient's pre-operative severity, so operating on a frail, complex patient no longer drags the number down; the score is built from a composite (mortality plus complications plus readmission) so a single dodge doesn't lift it; and the authority audits referral patterns for exactly the avoidance signature the old metric induced. When a cluster of surgeons starts routing borderline cases to a neighboring hospital, the monitor catches the drift and the risk model is revised. The score now rewards taking on and handling hard cases well — the outcome that was wanted all along.
How it works¶
- Specify the real outcome first, in words, before choosing any number. The metric is drafted as a proxy for that outcome, and the drafting question is always "what could raise this score without advancing the outcome?"
- Widen the distance a gamer must travel. Risk-adjust so easy inputs can't be selected for; combine several sub-measures into a composite so no single lever dominates; lag or hold out part of the measure so it can't be spot-optimized.
- Instrument for gaming after launch. Track not just the score but the ways it is moving — sudden distributional shifts, avoidance patterns, suspiciously round improvements — because a metric that was incentive-compatible on day one degrades as participants learn it.
- Re-tune on a schedule. Treat the formula as revisable, and change it when the monitor shows a game has opened, rather than defending a captured number.
Tuning parameters¶
- Proxy distance — how tightly the measure tracks the true outcome. Closer is harder to game but usually costlier to compute and slower to observe.
- Risk-adjustment granularity — how finely inputs are normalized. More granularity kills cream-skimming but adds model complexity and its own manipulable knobs.
- Composite breadth — how many sub-measures feed the score. Broader resists single-lever gaming but dilutes clarity and can hide a captured component.
- Formula transparency — how much of the scoring rule is published. Full disclosure builds legitimacy but hands would-be gamers the exact surface to optimize; partial concealment resists gaming but invites distrust.
- Refresh cadence — how often the rule is revised. Frequent updates outrun gamers but destabilize planning and can look arbitrary.
When it helps, and when it misleads¶
Its strength is that it lets a system publish and reward a number without the number rotting — it keeps a metric honest in the presence of people who are paid to bend it, and it converts "the surgeons are gaming us" from a scandal into a scheduled re-tune. It is the right mechanism precisely when a measure is unavoidable but a naive measure would be captured.
Its central failure is the one every metric shares: as soon as a measure becomes a target it ceases to be a good measure, because pressure flows straight into the proxy-outcome gap.[n1] Over-adjustment is its own trap — each correction adds a knob that becomes the next thing to game — and a composite can lend false confidence to a score that is captured on the component nobody is watching. The discipline that keeps it honest is to run the gaming monitor as a permanent fixture, not a launch check, and to keep the measured proxy as close as feasible to the outcome it stands in for rather than trusting a clever formula to hold forever.
How it implements the components¶
desired_outcome_specification— the rule is written from an explicit statement of the real outcome, so the score can be judged by how faithfully it tracks that outcome rather than by its own internal tidiness.failure_and_gaming_monitor— it instruments the deployed metric for the avoidance patterns, distributional jumps, and proxy-capture signatures that reveal a game has opened.adaptive_rule_update— the scoring formula is explicitly revisable and gets re-tuned when the monitor shows the current version has been learned.
It attaches no consequences to the score: it does not implement incentive_payoff_map or penalty_or_reward_rule — hanging pay on the outcome is Incentive Contract, and probabilistic inspection with penalties is Audit and Penalty System. This page is the measure; those siblings are the stakes.
Related¶
- Instantiates: Incentive-Compatible Rule Design — supplies the gaming-resistant measure the rest of the rule environment can reward or publish.
- Sibling mechanisms: Incentive Contract · Audit and Penalty System · Reputation-Weighted Participation · Blind or Randomized Review Rule · Mechanism Design Protocol · Deposit, Bond, or Stake · Matching Rule Design · Truthful Auction Mechanism
Editorial Notes¶
Form Classification¶
Form family: Rule, Policy & Commitment
Rationale: Scores behavior so the top score is earned by producing the real outcome, not by manipulating the measured proxy, and re-tunes as gaming emerges, making its operative form a standing constraint, permission, threshold, obligation, or conditional rule.
Independent corroboration: The frozen evidence defines Anti-Gaming Scoring Rule as 'Scores behavior so the top score is earned by producing the real outcome, not by manipulating the measured proxy, and re-tunes as gaming emerges', so its operative form is Rule, Policy & Commitment.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Economics & Finance
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Incentive and mechanism-design economics supplies the requirement that strategic participants maximize a score by producing the desired outcome rather than gaming its proxy.
Related originating lineages:
- Behavioral Economics — Observed strategic responses inform retuning beyond ideal rational models.
- Operations Research — Objective-function design and optimization expose proxy exploits.
- Public Administration & Policy — Goodhart's and Campbell's laws arose prominently around policy indicators.
- Statistics & Experimental Design — Measurement validity, risk adjustment, and composite design support robust scoring.
- Ethics of Technology & AI Governance — Metric gaming and algorithmic accountability motivate governance controls.
Review resolution: Mechanism-design economics is primary. Behavioral response, optimization, public performance management, risk adjustment, and metric governance materially shape an adaptive anti-gaming score; the generalized rule is an Encyclopedia synthesis with multi-domain reach.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Reconciled after independent review; high confidence.
Notes¶
[n1] Goodhart's Law — "when a measure becomes a target, it ceases to be a good measure" (the pointed restatement by Marilyn Strathern of Charles Goodhart's original observation about monetary indicators). Campbell's Law is the closely related version for social indicators. Anti-gaming scoring is the design response: hold the measure close to the outcome and monitor for the capture the law predicts. ↩