Metric Value Review¶
Procedure — instantiates Normative Assumption Explicitness
Examines what a metric rewards, ignores, normalizes, or sacrifices, especially when the metric is treated as objective evidence of success.
A Metric Value Review is a repeatable procedure aimed at exactly one thing: a single metric, KPI, ranking, or score that is being used as though it were an objective verdict on success. Its distinctive move is to split the seam the metric hides — between what the number measures (an empirical fact) and what someone has decided the number means (a value judgment about what ought to count as good). A metric of "minutes" measures minutes; the claim that fewer minutes is better care is a value choice wearing a number's clothes. The review names that choice, catalogs what the metric rewards, ignores, normalizes, and sacrifices, and fixes the scope in which its meaning legitimately holds. It stays deliberately narrow — one metric at a time — which is what separates it from a whole-surface audit.
Example¶
A hospital emergency department has made door-to-doctor time its headline quality metric, reported to the board as evidence the ER is improving. A Metric Value Review takes that one number apart. What it measures is real and unambiguous: median minutes from arrival to first clinician contact. What it is being used to mean is that speed equals ER quality — and that is a value assumption, not a measurement. The review then works the ledger. The metric rewards fast triage and quick hand-offs. It ignores diagnostic accuracy and whether the fast first contact was the right clinician. It normalizes fast-tracking simple cases to pull the median down. It sacrifices the complex patient whose long, correct workup makes the number look worse.
Then it fixes scope: door-to-doctor time is a reasonable operational flow signal, but its meaning does not extend to "clinical quality," and it should not silently govern individual nurses' evaluations. The deliverable is a short finding — this number measures flow, not quality; using it as the definition of success rewards speed over correctness — which is the classic shape of Goodhart's law biting a good measure the moment it becomes a target.[n1]
How it works¶
- State the measurement plainly. Write down what the number literally counts, stripped of interpretation.
- Name the success-value it stands for. Make explicit the "and therefore this is good" claim the number is being used to carry.
- Run the reward / ignore / normalize / sacrifice ledger. Catalog the behaviors the metric incentivizes and the values it quietly costs.
- Fix the scope of validity. Define the context where the number's value-meaning holds, and flag where it is being over-extended.
Tuning parameters¶
- Ledger granularity — how finely the rewarded/ignored/sacrificed effects are itemized. Finer catches gaming; coarser is faster.
- Scope tightness — how narrowly the metric's legitimate meaning is bounded. Tighter prevents over-reach but constrains reuse.
- Counter-metric proposal — whether the review stops at critique or proposes a balancing measure. Proposing adds value but adds work and new assumptions.
- Cadence — one-off versus recurring re-review as the metric's use spreads.
When it helps, and when it misleads¶
Its strength is puncturing the objective-looking number that has quietly redefined success — the single figure a board treats as truth. It is the sharpest tool against Goodhart-style drift, where optimizing the proxy corrodes the goal it was meant to stand for.
Its failure mode is metric whack-a-mole: every number can be criticized for what it omits, so an over-eager review can paralyze measurement entirely or be weaponized to kill any metric someone dislikes. The guarding discipline is to trigger the review where the definition of success itself is contested or consequential, not wherever a number is imperfect, and to pair critique with a workable alternative rather than leaving a vacuum.
How it implements the components¶
fact_value_boundary— its core operation: separates what the metric measures from the claim that the measured thing ought to count as success.competing_norm— the ignored-and-sacrificed column of the ledger names the credible values the metric trades away.scope_condition— bounds the context in which the metric's value-meaning legitimately holds, flagging silent over-extension.
It stays inside a single number; it does not run the whole-surface forensic sweep or test the organization's broader neutrality claim (legitimacy_check) — that breadth is Value Audit, its nearest analytical twin. Nor does it archive the finding (assumption_record, Decision Record).
Related¶
- Instantiates: Normative Assumption Explicitness — supplies the fact/value split and displaced-value ledger for a specific metric.
- Sibling mechanisms: Value Audit · Decision Record · Policy Rationale Statement · Stakeholder Deliberation · Ethics Checklist · Model Card Value Section · Values Statement · Ethical Impact Assessment
Editorial Notes¶
Form Classification¶
Form family: Assessment, Review & Assurance
Rationale: Metric Value Review operates as a bounded evaluation of existing evidence or work that produces a finding or disposition because it examines what a metric rewards, ignores, normalizes, or sacrifices, especially when the metric is treated as objective evidence of success.
Independent corroboration: The frozen evidence defines Metric Value Review as 'Examines what a metric rewards, ignores, normalizes, or sacrifices, especially when the metric is treated as objective evidence of success', so its operative form is Assessment, Review & Assurance.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Philosophy
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Universal
Rationale: Interrogating what a measure rewards and sacrifices is a normative and epistemological analysis.
Related originating lineages:
- Organizational & Management Science — Performance-management practice supplies the institutional setting where metrics become objectives.
- Sociology & Anthropology — For Metric Value Review, institutions, membership, social scale, norms, and collective meaning materially shaped the mechanism's characteristic form.
- Ethics of Technology & AI Governance — Algorithmic accountability made embedded metric values an applied governance concern.
Review resolution: Both independent reviews place the primary provenance in philosophy. The queued differences (alternate_origin_disagreement, domain_reach_disagreement) concern secondary metadata, not primary lineage. The final retains organizational_management, tech_ethics_ai_governance, sociology_anthropology only where a reviewer supplied a formative-lineage rationale; downstream use or broad applicability by itself is not treated as origin. origin_mode=cross_disciplinary_synthesis because the supplied rationales identify formative contributions that are composed in the mechanism's present form. domain_reach=universal records established application breadth separately from provenance. confidence=medium preserves the more cautious evidence assessment. encyclopedia_synthesis=true records whether either reviewer identified deliberate corpus-level composition.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Reconciled after independent review; medium confidence.
Notes¶
[n1] Goodhart's law — "when a measure becomes a target, it ceases to be a good measure." A Metric Value Review is a direct antidote: it re-exposes the value assumption a targeted metric has come to stand for, before optimization hollows out the goal. ↩