Deployment Impact Dashboard¶
Metric / dashboard — instantiates Malleability Window Governance
Tracks outcome, harm, dependency, and lock-in signals during staged deployment.
The Deployment Impact Dashboard is a live instrument panel that, during a staged rollout, tracks four signal families side by side: outcomes achieved, harms produced (including delayed and distributed ones), how deeply the system is being depended on, and how expensive it would now be to switch away. Its defining move is to place a rising lock-in curve next to the benefit curve on the same screen. Most monitoring shows only whether the system is performing; this one refuses to let good performance mask a window quietly closing, by making dependency and switching cost first-class metrics that climb in plain sight. It answers the archetype's core discipline — track knowledge and malleability together — in the form of a dashboard.
Example¶
A hospital rolls out an AI sepsis-alert model ward by ward. The dashboard tracks four things at once. Outcomes: alert precision and time-to-antibiotics. Harms: alert-fatigue override rates and missed atypical presentations, disaggregated by patient subgroup so a harm concentrated in one group cannot hide in the average. Dependency: the share of clinical workflow now written to assume the alert exists. Switching cost: the tested time and staffing needed to revert wards to the manual escalation protocol. Three months in, the panel shows precision improving slowly while switching cost climbs steeply — dependency is outpacing proven benefit. Nothing has gone wrong operationally, but the picture forces a governance conversation the raw performance number would never have prompted.
How it works¶
- Leading lock-in metrics beside lagging outcome metrics. Benefit shows up late; dependency and switching cost show up early. Pairing them is the point.
- Harm watch tuned for the hard cases. The instrumentation is built to catch delayed, externalized, and vulnerable-subgroup harms — disaggregated, not aggregated — rather than only headline performance.
- Dependency and switching cost as tracked quantities. Unusually, these are measured continuously (integrations built on the system, cost-to-revert), not assumed.
- Displays thresholds, does not pull them. The panel can flag a breach; the authority to act on it lives in other mechanisms.
- Staged, not steady-state. It is read during a rollout that is still advancing rung by rung, so its job is to inform the next progression decision, not to certify a finished system as healthy.
Tuning parameters¶
- Metric set — which harm and dependency proxies are instrumented. Broader coverage catches more but dilutes attention and costs engineering.
- Refresh cadence — real-time versus periodic. Faster feedback catches fast-moving harm but amplifies noise and over-reaction.
- Subgroup disaggregation depth — how finely harms are split by population. Deeper cuts surface concentrated harms but raise small-sample false alarms.
- Alerting threshold — what merely displays versus what raises an alarm. Sensitive alarms catch problems early at the cost of fatigue.
When it helps, and when it misleads¶
Its strength is keeping lock-in visible alongside benefit, so no one can point at green outcome tiles while dependency silently forecloses the exit. Its failure mode is Goodhart's law[n1]: once a dashboard metric becomes the target, operators optimize the tile rather than the outcome, and any harm that isn't instrumented is treated as harm that isn't happening. A classic misuse is an all-green dashboard waved as proof of "responsible deployment" while the one consequence that matters was never a column. The guarding discipline is to pair the panel with an external reporting channel that catches off-dashboard harm, and to rotate and re-derive metrics so they cannot ossify into gameable targets.
How it implements the components¶
deployment_telemetry_and_harm_watch— the outcome and harm instrumentation, deliberately built to surface delayed, distributed, and subgroup-specific harms rather than aggregate performance alone.lock_in_cost_monitor— dependency depth and switching cost tracked as live, first-class metrics rising next to benefit.
It displays signals but does not gather them from the people the system acts on — reports from affected parties flow through stakeholder_impact_forum, owned by Stakeholder Harm Reporting Channel; and it holds no moratorium_trigger to halt on a breach, which is Pause or Moratorium Trigger Protocol's.
Related¶
- Instantiates: Malleability Window Governance — the dashboard is how the archetype's "track knowledge and lock-in together" invariant becomes an operational instrument.
- Sibling mechanisms: Adaptive Stage-Gate Protocol · Collingridge Curve Workshop · Exit and Interoperability Rule · Pause or Moratorium Trigger Protocol · Post-Pilot Lock-In Audit · Regulatory or Operational Sandbox · Reversibility Horizon Review · Stakeholder Harm Reporting Channel · Sunset Clause with Renewal Hearing
Editorial Notes¶
Form Classification¶
Form family: Monitoring, Sensing & Alerting
Rationale: Deployment Impact Dashboard operates as an ongoing sensing arrangement that repeatedly observes actual state and surfaces changes or alerts because it tracks outcome, harm, dependency, and lock-in signals during staged deployment.
Independent corroboration: The frozen evidence defines Deployment Impact Dashboard as 'Tracks outcome, harm, dependency, and lock-in signals during staged deployment', so its operative form is Monitoring, Sensing & Alerting.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Ethics of Technology & AI Governance
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Technology governance cohered staged-deployment monitoring that places benefits, harms, dependency, and lock-in on the same oversight surface.
Related originating lineages:
- Data Science & Analytics — Model monitoring and staged evaluation supplied live outcome, drift, and performance measures for the dashboard.
- Organizational & Management Science — Change management supplied staged rollout dashboards and contingency triggers.
- Public Administration & Policy — Impact assessment supplied distributional-harm and accountability indicators.
Review resolution: Technology governance cohered staged-deployment monitoring that places benefits, harms, dependency, and lock-in on the same oversight surface. The retained alternate lineages materially shaped the mechanism's form.
Attribution caveat: The joint benefit-harm-lock-in dashboard is an encyclopedia synthesis rather than a single established disciplinary instrument.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Reconciled after independent review; medium confidence.
Notes¶
[n1] Goodhart's law — "when a measure becomes a target, it ceases to be a good measure." A governance dashboard is unusually exposed to it, because the operators being watched are also the ones who can move the needles; rotating metrics and pairing them with un-gameable external reports is the standard defense. ↩