Risk Score Threshold Recalibration¶
Model-governance procedure — instantiates Adaptive Threshold Recalibration
Moves the score boundary that routes cases to auto-approve, review, or deny when a deployed model's population or performance has drifted, keeping a human channel for contested cases.
Risk Score Threshold Recalibration governs the operating cutoffs of a live scoring model — the boundaries that route each case to auto-approve, to manual review, or to deny. What makes it its own mechanism is the pairing of two things: it is a governed procedure (a documented evidence bar and formal sign-off, not an ad-hoc tweak), and it deliberately preserves a human override channel for cases near or contesting the boundary. It is triggered when the deployed model's population or performance drifts, so the same score now sorts cases differently than intended. Recalibration re-derives the routing boundaries under governance while keeping a person in the loop where the stakes to an individual are highest.
Example¶
A consumer lender uses a default-risk model with three bands: auto-approve below a score, route a middle band to human underwriting, deny above. Over a year the applicant population shifts — more thin-file applicants whose scores land lower — so cases that should get a closer look are sliding into auto-approve. Monitoring flags the population drift. Risk Score Threshold Recalibration re-derives the band boundaries under the model-governance procedure: a documented evidence review, sign-off from the model-risk function, and adverse-action reason codes updated to match. Crucially, the middle manual-review band is preserved and its width tuned, so underwriters still see borderline cases — and an override channel lets them reverse an edge decision[1], with each reversal logged and fed back as evidence for the next review.
How it works¶
The distinctive combination is governance plus recourse. The recalibration rule is explicit about what evidence justifies moving the routing cutoffs (sustained drift, not a noisy month) and who must approve — this is a controlled change to a consequential automated decision, not a dial anyone can turn. And because the boundary decides individual outcomes, a human override channel is wired in by design: contested or edge cases reach a person, and their decisions become logged feedback that informs the next recalibration. The mechanism operates on the deployed score's boundaries; it does not re-fit the model.
Tuning parameters¶
- Band boundaries — the approve / review / deny cutoffs; the direct levers over routing and volume.
- Manual-review band width — how wide the human-in-the-loop zone is; wider catches more edge cases but costs underwriter time.
- Override authority + logging — who may reverse an automated call, and how reversals are recorded and fed back.
- Recalibration trigger sensitivity — how much drift, sustained how long, licenses a change; guards against chasing noise.
- Adverse-action handling — how reason codes and notifications track the moved boundary so denials stay explainable.
When it helps, and when it misleads¶
Its strength is keeping a live model's decisions matched to the current population while preserving human recourse and an audit trail — the governance that consequential automated decisions demand. Its classic misuse is recalibrating toward a business target — loosening the approve band to chase volume — and letting "the population drifted" launder a decision made for growth. Because the override channel and the routing cutoffs are powerful, the temptation is to move them quietly. The discipline is the governed evidence bar and the logged override feedback: a boundary change must be justified by drift evidence and its downstream effects watched, not reverse-engineered from a volume goal.
How it implements the components¶
recalibration_rule— it defines the governed rule converting drift evidence into revised routing cutoffs, including the sufficiency-of-evidence and sign-off bar.human_override_channel— it wires in the path for contested and edge cases to reach a person, and feeds their reversals back as evidence.monitored_variable_or_score— it operates on the deployed risk score whose boundaries are being moved, tracking that the score is measured consistently.
It does not verify the score is still calibrated before the move (drift_detection_signal) — Calibration Curve Review does — nor does it test the candidate cutoff head-to-head (validation_and_backtesting_plan), which Champion / Challenger Threshold Test owns.
Related¶
- Instantiates: Adaptive Threshold Recalibration — it is the model-governance instance: moving a deployed score's operating cutoffs under control, with recourse.
- Consumes: Calibration Curve Review — the prior check that the score is still honest before its boundary is moved.
- Sibling mechanisms: Champion / Challenger Threshold Test · Calibration Curve Review · Receiver Operating Characteristic Review · Precision / Recall Tradeoff Review · Threshold Versioning Register
Editorial Notes¶
Form Classification¶
Form family: Analysis, Modeling & Optimization
Rationale: Risk Score Threshold Recalibration operates by uses sustained drift evidence and realized outcomes to compute revised routing cutoffs. That concrete deployed or enacted form is Analysis, Modeling & Optimization under the frozen taxonomy.
Nearest alternative: Rule, Policy & Commitment — Although Rule, Policy & Commitment can support this mechanism, the frozen evidence makes its operative form the act that uses sustained drift evidence and realized outcomes to compute revised routing cutoffs; the alternative is therefore secondary rather than defining.
Review outcome: Adjudicated after independent review; high confidence.
Origin Attribution¶
Primary origin: Data Science & Analytics
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Moving deployed model thresholds under population and performance drift is a machine-learning operations practice.
Related originating lineages:
- Computer Science & Software Engineering — Computer science and software-engineering practice supplies a parallel or contributing lineage for the mechanism's defining operation: moves the score boundary that routes cases to auto-approve, review, or deny when a deployed model's population or performance has drifted, keeping a human channel for contested cases.
- Statistics & Experimental Design — Calibration and decision-theoretic cutoff selection supply the inferential basis.
- Ethics of Technology & AI Governance — Contestability and human review materially constrain threshold changes.
Review resolution: Both blind reviewers agree that data_science is the primary historical origin. Explicit reconciliation of alternate origin disagreement, domain reach disagreement starts from reviewer_a’s mechanism-specific evidence: Moving deployed model thresholds under population and performance drift is a machine-learning operations practice. Reviewer A proposed alternates=statistics_experimental_design, tech_ethics_ai_governance, origin_mode=cross_disciplinary_synthesis, domain_reach=multi_domain, and encyclopedia_synthesis=true; reviewer B proposed alternates=computer_science, statistics_experimental_design, origin_mode=cross_disciplinary_synthesis, domain_reach=specialized, and encyclopedia_synthesis=true. The final record retains every independently supported alternate from either review (statistics_experimental_design, tech_ethics_ai_governance, computer_science) without an arbitrary cap, selects origin_mode=cross_disciplinary_synthesis to represent the combined lineage evidence, and keeps domain_reach=multi_domain and encyclopedia_synthesis=true from the more mechanism-specific assessment. Present-day transfer is recorded as reach and is not treated as proof of historical origin.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Reconciled after independent review; high confidence.
References¶
[1] Board of Governors of the Federal Reserve System and Office of the Comptroller of the Currency. Supervisory Guidance on Model Risk Management. SR Letter 11-7 / OCC Bulletin 2011-12, 2011. Recognizes expert-judgment overrides in which model users ignore, alter, or reverse a model output. registry ↩