Proxy Drift Dashboard¶
Metric / dashboard — instantiates Abstraction–Substrate Traceability Guardrail
Monitors divergence between proxy indicators and direct substrate checks over time.
A Proxy Drift Dashboard is a running instrument that tracks, quantitatively and continuously, whether a proxy still moves with the underlying condition it stands for. Its defining idea is the paired series: it holds the cheap proxy indicator next to a periodic, expensive direct measurement of the substrate, and watches the gap between them as a trend. When the two move together, the proxy is earning its keep; when they separate — the proxy climbing while the substrate stalls, or the proxy flat while the substrate slips — the dashboard raises the divergence as a monitored, thresholded signal. It is not a case adjudicator and not a scheduled review; it is an always-on metric whose job is to catch the slow, aggregate decoupling of indicator from reality before anyone has optimized the number into a lie.
Example¶
A municipal water utility uses continuous turbidity sensors as a proxy for microbial contamination — turbidity is cheap and instant, lab culturing of pathogens is slow and costly. Operations, alarms, and public reporting all run off the sensor. A Proxy Drift Dashboard pairs the two: the live turbidity series against the results of the routine lab samples pulled twice a week. For a year the correlation is tight. Then the dashboard's divergence panel starts to climb — lab pathogen counts creeping up on days turbidity reads clean.
Investigation finds the cause: a new upstream source contributes contamination that is biologically active but optically clear, so the proxy no longer sees it. The dashboard did not diagnose the source and did not decide to retire the sensor; it did the one thing it exists for — it made the decoupling visible as a trend, early, so the utility stopped treating "low turbidity" as "safe water" before the gap caused harm. The proxy had quietly stopped tracking the substrate, and only a paired, continuous comparison could show it.
How it works¶
The dashboard is continuous and comparison-driven:
- Pair proxy with substrate truth. Instrument the cheap indicator and schedule periodic direct measurements of the underlying condition; the direct check is the anchor the proxy is graded against.
- Track the gap as a series. Plot divergence over time — correlation decay, growing residual, or a widening ratio — so slow decoupling shows as a trend, not a one-off surprise.
- Threshold and alert. Define how much sustained divergence counts as drift, and surface it as a signal, separating normal noise from a proxy that has stopped tracking.
- Watch for gaming, not just decay. Distinguish natural drift from the case where actors are optimizing the proxy directly — the tell is the proxy improving while the substrate does not.
The distinguishing discipline is quantitative continuity: it measures the aggregate relationship over time rather than ruling on any single instance, and its output is a monitored gap, not a disposition.
Tuning parameters¶
- Substrate sampling rate — how often the expensive direct check is taken. More frequent anchoring catches drift sooner but costs more; sparse anchoring saves money but lets the gap grow unseen.
- Drift threshold — how much sustained divergence triggers an alert. Tight thresholds catch decoupling early but cry wolf on noise; loose ones fire late.
- Window length — how much history the trend averages over. Long windows are stable but slow to react; short ones are twitchy but timely.
- Gaming sensitivity — whether the dashboard specifically flags proxy-up/substrate-flat patterns. Tuning for gaming catches manipulation but needs the substrate series clean enough to trust.
When it helps, and when it misleads¶
Its strength is that it enforces the archetype's hardest invariant — no proxy-only optimization — with data: it refuses to let "the number went up" stand as proof the condition improved, because it is watching the condition directly alongside. It is the operational answer to Goodhart's law, the rule that a measure adopted as a target ceases to be a good measure once people optimize it.[n1] A live paired series is the earliest, cheapest way to catch that decoupling.
It misleads when the "direct" substrate anchor is itself a proxy — the failure of substrate fetishism, where a slightly-less-abstract measurement is trusted as ground truth though it too is sampled and instrumented. A dashboard comparing two proxies can show reassuring agreement while both drift together. It also only detects divergence, never explains it, and a jittery threshold breeds alert fatigue that gets the whole panel ignored. The classic misuse is celebrating a stable gap as validation while the substrate anchor quietly degraded. The guarding discipline is to keep the anchor as close to real direct observation as feasible, re-verify the anchor's own integrity, and treat a divergence signal as the start of an investigation, not a conclusion.
How it implements the components¶
Proxy Drift Dashboard fills the monitoring components — the guardrail's continuous check that the proxy still tracks reality:
proxy_optimization_monitor— it is the standing watch against proxy-only optimization: by holding the substrate series beside the proxy, it exposes any gain in the number that the underlying condition did not earn.substrate_divergence_channel— its thresholded divergence signal is the aggregate, quantitative path by which a decoupling of proxy from substrate reaches attention.
It does not adjudicate individual contradicting cases or give affected subjects standing (affected_subject_review_channel, revision_or_retirement_path) — that is Counterexample Case Review, which rules on specific instances where this dashboard tracks the aggregate gap; nor does it run the calendar-scheduled requalification (re_grounding_cadence_and_trigger) — that is Re-grounding Review Cadence.
Related¶
- Instantiates: Abstraction–Substrate Traceability Guardrail — the dashboard enforces the no-proxy-only-optimization invariant with a live paired series.
- Sibling mechanisms: Counterexample Case Review · Re-grounding Review Cadence · Point-of-Use Reification Warning · Map–Territory Review Checklist · Model Card or Datasheet Linkage
Editorial Notes¶
Form Classification¶
Form family: Monitoring, Sensing & Alerting
Rationale: Proxy Drift Dashboard operates as ongoing observation, sensing, or alerting that detects and surfaces state without itself executing the response because it monitors divergence between proxy indicators and direct substrate checks over time.
Independent corroboration: The frozen evidence defines Proxy Drift Dashboard as 'Monitors divergence between proxy indicators and direct substrate checks over time', so its operative form is Monitoring, Sensing & Alerting.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Data Science & Analytics
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Monitoring production proxy divergence against direct checks is a data-science and model-monitoring artifact.
Related originating lineages:
- Statistics & Experimental Design — Longitudinal validity assessment supplies the inferential basis for declaring drift.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Independent reviewer agreement; high confidence.
Notes¶
[n1] Goodhart's law — "when a measure becomes a target, it ceases to be a good measure." Once a proxy is optimized for its own sake, it decouples from the condition it once tracked. The dashboard is built to catch that decoupling in the act, by watching the substrate the proxy was supposed to represent. ↩