Skip to content

Confidence Threshold Table

Decision-threshold artifact — instantiates Predictive Residual Processing

A maintained lookup table that turns model confidence and residual size into an action — pass, review, or escalate — indexed by stage and risk level.

A residual-processing loop constantly asks how much surprise, or how little confidence, is enough to act? — and if that answer lives scattered in code and individual judgment, it drifts and can't be audited. Confidence Threshold Table is the reference artifact that codifies those cutoffs: for each stage and risk tier, how much residual (or how little confidence) triggers pass / continue / review / escalate, and what tolerated error budget sits behind each line. It computes nothing and detects nothing; it is the governed rulebook everything else reads to decide what a given confidence means for action — a single, versioned source of truth that separates what the model thinks from what we do about it.

Example

A content-moderation classifier outputs a confidence that a post violates policy. The table maps the bands: ≥0.98 → auto-remove; 0.70–0.98 → route to a human reviewer; <0.70 → auto-approve — with deliberately tighter cutoffs for high-risk categories, where borderline cases route to a human at much lower confidence. Behind each band sits an explicit budget: the tolerated false-allow rate for that tier. When the platform's risk appetite shifts — say, tightening on a sensitive category — someone edits the table and versions the change; the model is untouched. Reviewers, auditors, and the pipeline all read the same numbers.

How it works

  • Enumerate stages × risk tiers. The rows and columns are the situations where a suppress/act decision is made.
  • Set the cutoffs and the action for each cell — the confidence or residual level that maps to pass, review, or escalate.
  • Record the error budget behind each cutoff, so a threshold is justified by a tolerated error rate rather than a hunch.
  • Version it. Every change is dated and attributable, because the thresholds are the policy.

Tuning parameters

  • Number of bands and cutoff values — each cutoff picks a point on the precision/recall trade for that tier.
  • Risk-tier granularity — more tiers fit the response to consequence but add maintenance and edge-case ambiguity.
  • Tolerated error budget per tier — the honest statement of how much residual you're willing to let through.
  • Escalation routing — where a "review" or "escalate" outcome goes.
  • Review and versioning cadence — how often the table is revisited so it doesn't ossify.

When it helps, and when it misleads

Its strength is making the suppress-or-act policy explicit, auditable, and adjustable without touching the model — you can change the organization's risk posture by editing a table.

Its failure modes: a static table silently drifts out of step with a changing model or world, and published cutoffs become targets people optimize toward rather than honest boundaries — the moment a threshold is a KPI, behavior bends to clear it.[1] The classic misuse is setting thresholds to hit a throughput or queue-size target rather than an error target, quietly trading safety for volume. The discipline: tie every cutoff to a measured error budget (from backtesting), review on a cadence, and version each change so drift is visible.

How it implements the components

  • residual_error_budget — the table is where the tolerated residual/error per stage and tier is written down and governed.
  • confidence_and_uncertainty_state — it operates on confidence levels, mapping bands of the uncertainty state to concrete actions.

It does not produce the confidence estimate — Bayesian Model Update generates the uncertainty this table merely thresholds — nor apply the cutoffs in-line to each error (that's Precision-Weighted Error Gate), nor route a triggered escalation to a response (that's Surprise-to-Action Bridge).

Notes

There is a clean division of labor worth stating: Bayesian Model Update produces a confidence/uncertainty; this table decides what confidence levels do. Keeping the two apart lets you retune the policy (edit the table) without re-deriving the model, and vice versa.

References

[1] Goodhart's law — "when a measure becomes a target, it ceases to be a good measure." Published decision thresholds are especially prone to this; tying cutoffs to an underlying error budget and reviewing them on a cadence is the standard guard. ↩