Skip to content

Risk Score Threshold Recalibration

Model-governance procedure — instantiates Adaptive Threshold Recalibration

Moves the score boundary that routes cases to auto-approve, review, or deny when a deployed model's population or performance has drifted, keeping a human channel for contested cases.

Risk Score Threshold Recalibration governs the operating cutoffs of a live scoring model — the boundaries that route each case to auto-approve, to manual review, or to deny. What makes it its own mechanism is the pairing of two things: it is a governed procedure (a documented evidence bar and formal sign-off, not an ad-hoc tweak), and it deliberately preserves a human override channel for cases near or contesting the boundary. It is triggered when the deployed model's population or performance drifts, so the same score now sorts cases differently than intended. Recalibration re-derives the routing boundaries under governance while keeping a person in the loop where the stakes to an individual are highest.

Example

A consumer lender uses a default-risk model with three bands: auto-approve below a score, route a middle band to human underwriting, deny above. Over a year the applicant population shifts — more thin-file applicants whose scores land lower — so cases that should get a closer look are sliding into auto-approve. Monitoring flags the population drift. Risk Score Threshold Recalibration re-derives the band boundaries under the model-governance procedure: a documented evidence review, sign-off from the model-risk function, and adverse-action reason codes updated to match. Crucially, the middle manual-review band is preserved and its width tuned, so underwriters still see borderline cases — and an override channel lets them reverse an edge decision, with each reversal logged and fed back as evidence for the next review.[^drift]

How it works

The distinctive combination is governance plus recourse. The recalibration rule is explicit about what evidence justifies moving the routing cutoffs (sustained drift, not a noisy month) and who must approve — this is a controlled change to a consequential automated decision, not a dial anyone can turn. And because the boundary decides individual outcomes, a human override channel is wired in by design: contested or edge cases reach a person, and their decisions become logged feedback that informs the next recalibration. The mechanism operates on the deployed score's boundaries; it does not re-fit the model.

Tuning parameters

  • Band boundaries — the approve / review / deny cutoffs; the direct levers over routing and volume.
  • Manual-review band width — how wide the human-in-the-loop zone is; wider catches more edge cases but costs underwriter time.
  • Override authority + logging — who may reverse an automated call, and how reversals are recorded and fed back.
  • Recalibration trigger sensitivity — how much drift, sustained how long, licenses a change; guards against chasing noise.
  • Adverse-action handling — how reason codes and notifications track the moved boundary so denials stay explainable.

When it helps, and when it misleads

Its strength is keeping a live model's decisions matched to the current population while preserving human recourse and an audit trail — the governance that consequential automated decisions demand. Its classic misuse is recalibrating toward a business target — loosening the approve band to chase volume — and letting "the population drifted" launder a decision made for growth. Because the override channel and the routing cutoffs are powerful, the temptation is to move them quietly. The discipline is the governed evidence bar and the logged override feedback: a boundary change must be justified by drift evidence and its downstream effects watched, not reverse-engineered from a volume goal.

How it implements the components

  • recalibration_rule — it defines the governed rule converting drift evidence into revised routing cutoffs, including the sufficiency-of-evidence and sign-off bar.
  • human_override_channel — it wires in the path for contested and edge cases to reach a person, and feeds their reversals back as evidence.
  • monitored_variable_or_score — it operates on the deployed risk score whose boundaries are being moved, tracking that the score is measured consistently.

It does not verify the score is still calibrated before the move (drift_detection_signal) — Calibration Curve Review does — nor does it test the candidate cutoff head-to-head (validation_and_backtesting_plan), which Champion / Challenger Threshold Test owns.