Skip to content

Escalation Ladder Integrity Audit

Integrity audit — instantiates Signal Value Preservation

Periodically checks that each rung of a severity or priority ladder still maps to a genuinely distinct level of seriousness, catching tier inflation before every case creeps to the top rung.

The Escalation Ladder Integrity Audit is a recurring review of a graded signal system — SEV-1 through SEV-5, P0 through P4, colour-coded threat levels — that tests whether the rungs still separate. The one idea that makes it this mechanism is that it audits the ladder as a whole for grade inflation: not whether one firing was justified, and not how precise the signal is, but whether the tiers as a set have quietly collapsed into each other as cases crept upward over time. Its object is the portfolio of levels and their history, and its question is diagnostic separation — can a receiver still tell why a case on this rung differs from one on the rung below?

Example

The United States once ran the Homeland Security Advisory System — the colour-coded terrorism scale from green to red. For years it sat at Yellow ("Elevated") or Orange ("High") and never dropped to the lower rungs, so the levels stopped informing anyone; the system was retired in 2011. An integrity audit is precisely what would have caught the collapse in advance: map the level-settings over the prior years, show that the bottom rungs were never used and the top was crowding, and test whether "Orange" was ever attached to distinguishably different evidence and response than "Yellow." The finding would be blunt — the five tiers had degraded into effectively one — and the remedy would be to recalibrate the thresholds or retire the scale rather than keep flying a flag that no longer sorted anything.

How it works

The audit assembles the full portfolio of tiers and their firing histories and then tests two things. Separation: does each tier's triggering evidence and required response genuinely differ from its neighbours — because two rungs that trigger the same action are not two rungs. Distribution drift: are cases migrating upward over time, so the top tiers are crowding and the bottom ones going unused? It flags collapsed or dead rungs and hands the correction to recalibration or redesign; it diagnoses, it does not itself re-set the thresholds.

Tuning parameters

  • Audit cadence — how often the ladder is reviewed. Frequent audits catch drift early but cost analyst time; rare ones let inflation entrench before anyone looks.
  • Separation metric — how "distinct" is judged: distinct evidence, distinct required response, or distinct outcome. Response-based separation is the most honest test — tiers that trigger identical action are not really separate.
  • Drift threshold — how much upward migration counts as inflation worth flagging versus normal variation.
  • Scope — a single ladder or the whole family across the organisation, which also catches cross-ladder inconsistency (one team's SEV-2 is another's SEV-4).

When it helps, and when it misleads

Its strength is that it catches the specific, slow failure where grades creep upward until the top rung is the default and the ladder no longer sorts anything — the failure a single-firing check can never see. Its failure modes are that the audit can find inflation but has no power to fix it (it must hand off to recalibration), and that a purely statistical drift test can misread a genuine rise in real severity as inflation. It is misused to justify raising thresholds in a way that hides real risk — declaring true emergencies "over-classified." The discipline that guards against this is to judge separation by required response, corroborate drift against ground-truth outcomes, and route findings to recalibration rather than acting on them unilaterally.[1]

How it implements the components

  • signal_portfolio_map — its core: it lays out the full set of tiers and their firing histories as one map, so collapse, crowding, and dead rungs become visible at a glance.
  • evidence_and_specificity_rule — it tests that each rung is tied to distinguishably different evidence and category boundaries; a rung that isn't fails the audit.

It does not re-set the thresholds it finds broken (renewal_expiry_and_retirement_rule) — that's Severity Tier Recalibration — nor measure per-signal precision (signal_base_rate_monitor) — that's Base-Rate and Precision Dashboard.

References

[1] The Homeland Security Advisory System — the U.S. colour-coded terrorism threat scale (2002–2011). Because the level almost never left the middle of its range, the tiers stopped distinguishing anything and the system was replaced. It is a textbook case of tier collapse, the exact failure this audit exists to catch before it becomes irreversible. ↩