Incentive Impact Review¶
Structured review — instantiates Proxy–Target Divergence Detection and Recalibration
Maps the rewards, sanctions, and optimization pressure acting on a proxy to anticipate where actors will game the measure and hollow out its link to the target.
Incentive Impact Review is the loop's anticipatory mechanism. Instead of waiting for data to show a break, it reasons forward from the incentive structure: who is rewarded or punished by this proxy, how much pressure are they under, and what is the cheapest way for them to move the number without moving the target? Its defining move is to treat the measured actors as intelligent adaptive agents and to red-team the metric from their point of view — to find, before the data does, the gap between "improve the proxy" and "improve the target." Where statistical monitors detect gaming only after it has already distorted the signal, this review predicts which proxies are structurally vulnerable and which link assumptions the pressure is most likely to snap. It is analysis and imagination, not measurement.
Example¶
An ambulance service is judged on a single headline proxy for care quality: the share of emergency calls reached within eight minutes. An incentive impact review sits down before the next reporting cycle and maps the pressure. Station managers' funding and public league-table position ride on this number. The reviewers then think like the managers: what moves the eight-minute figure cheaply? They surface a list of gaming pathways that keep the metric green while care does not improve — dispatching the nearest vehicle regardless of whether it carries the right equipment, "stopping the clock" by logging arrival at the scene entrance rather than at the patient, reclassifying borderline calls into lower-urgency categories that fall outside the target. For each, they trace the consequence back to patients: a defibrillator-less first responder arriving fast helps the number and not the cardiac-arrest victim. The review's output is not a verdict that gaming is happening — it is a ranked map of where the incentive most threatens the proxy-target link, telling the audit and dashboard exactly where to look.
How it works¶
- Enumerate the pressures. List every reward, sanction, ranking, funding rule, promotion, and automated optimizer that acts on the proxy, and who feels each.
- Adopt the agent's view. For each pressured actor, brainstorm the lowest-effort ways to move the proxy that do not move the target — teaching to the test, definitional reclassification, effort reallocation, superficial compliance.
- Re-examine the link assumption under pressure. Take the written reason the proxy is supposed to track the target and ask which gaming pathway would break it while leaving the number intact.[n1]
- Trace consequences. Follow each plausible gaming route to the downstream harm it would cause, so the risk is expressed in target terms, not metric terms.
- Rank and route. Order the vulnerabilities by likelihood and stakes and hand them to the mechanisms that can watch for or confirm them.
Tuning parameters¶
- Adversarial depth — how hard the reviewers try to break the metric, from a quick checklist to a full adversarial workshop. Deeper reviews find subtler gaming but cost expert time and can breed false alarm.
- Actor coverage — how many distinct pressured parties are modeled. Narrow coverage is fast but misses gaming by unmodeled actors (vendors, intermediaries, upstream data enterers).
- Framing stance — punitive ("catch cheaters") versus systemic ("the incentive made this rational"). A systemic frame surfaces far more real pathways because it doesn't require anyone to be a villain.
- Review trigger — one-off at proxy launch, on every incentive change, or on a fixed cadence. Re-running when incentives change is the highest-value cadence.
- Consequence horizon — how far downstream harms are traced; longer horizons catch slow, diffuse damage but grow speculative.
When it helps, and when it misleads¶
Its strength is foresight and framing. It is the only mechanism here that can flag a decoupling before it exists in the data, and by naming pressures explicitly it turns "the team is cheating" into "the incentive structure makes cheating rational," which is both fairer and more fixable. It is indispensable exactly when a proxy is rewarded, published, or optimized — the conditions under which Goodhart's Law bites.[n1]
Its failure mode is that it produces hypotheses, not evidence. A vivid gaming story can feel like proof and provoke an overreaction against actors who never gamed anything, or it can miss the one pathway the reviewers lacked the domain knowledge to imagine. Framed punitively, it also breeds defensiveness that drives real gaming underground. The classic misuse is treating the review's ranked risk list as a finding of misconduct rather than a targeting map for real checks. The guarding discipline is to route every hypothesized vulnerability to an independent measurement — a targeted audit or shadow channel — for confirmation, and to keep the review's tone systemic so people surface pressures honestly instead of hiding them.
How it implements the components¶
use_and_pressure_map— its core product: an explicit map of how institutional rewards, sanctions, and optimization act on the proxy and on whom.harm_and_consequence_monitor— by tracing each gaming pathway to its downstream damage, it specifies the target-side harms worth watching for.proxy_target_link_assumption— it stress-tests the recorded reason the proxy should track the target, asking which pressure would sever that reason while leaving the number intact.
This review reasons about incentives; it does not statistically detect a break — that divergence_sentinel role is Drift and Change-Point Detection — nor does it measure the target on a sample — that independent_target_check belongs to Holdout Ground-Truth Audit.
Related¶
- Instantiates: Proxy–Target Divergence Detection and Recalibration — it supplies the pressure map that tells the rest of the loop where decoupling is most likely.
- Sibling mechanisms: Drift and Change-Point Detection · Holdout Ground-Truth Audit · Proxy Retirement Decision Record · Proxy–Target Correlation Refresh · Reference-Standard Recalibration Review · Sentinel Outcome Dashboard · Shadow Target Measurement · Triangulated Proxy Panel
Editorial Notes¶
Form Classification¶
Form family: Assessment, Review & Assurance
Rationale: Incentive Impact Review operates as a bounded evaluation of existing evidence or work that produces a finding or disposition because it maps the rewards, sanctions, and optimization pressure acting on a proxy to anticipate where actors will game the measure and hollow out its link to the target
Independent corroboration: The frozen evidence defines Incentive Impact Review as 'Maps the rewards, sanctions, and optimization pressure acting on a proxy to anticipate where actors will game the measure and hollow out its link to the target', so its operative form is Assessment, Review & Assurance.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Economics & Finance
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Anticipating how actors optimize a rewarded proxy is an incentive and principal-agent analysis, with Goodhart-style target distortion.
Related originating lineages:
- Organizational & Management Science — Performance-management audits materially institutionalized review of gaming and countermetrics.
- Public Administration & Policy — Campbell's Law and policy-target gaming provide an independent public-governance lineage.
Review resolution: Both reviewers independently assign economics_finance as the primary originating domain, so that shared primary is retained. Alternate domains are the union of reviewer-identified formative or independently originating lineages; later application settings alone are excluded. The final form materially composes methods or concepts from more than one formative domain. It has established independent use across several domains, but that does not make it domain-free. The encyclopedia entry makes that composition explicit.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Reconciled after independent review; high confidence.
Notes¶
[n1] Goodhart's Law — "when a measure becomes a target, it ceases to be a good measure" — and its close cousin Campbell's Law describe exactly the decoupling this review anticipates: the act of rewarding a proxy invites optimization that improves the metric without improving the target. The review is the deliberate attempt to predict those optimizations in advance. ↩a ↩b