Tensions in Practice: Fewer false alarms in tension with fewer misses¶
Four invented detector observations
A detector gives higher scores to observations it considers more likely to contain a signal. In this toy, N1 and N3 have no signal; S2 and S4 do, and the digit is the score. Flagging scores of at least 2 catches both signals but also flags N3. Raising the cutoff to 4 removes that false alarm while missing S2. The same evidence now makes a different tradeoff.
Avoid needless alerts
Leave signal-absent observations unflagged.
Catch the signal
Flag signal-present observations when action matters.
Why these aims pull against each other
Moving a cutoff changes which scores trigger action. It does not improve the separation already present in those scores.
Choose an arrangement to see what changes and what remains difficult.
Arrows express the declared relations, not measured effect sizes. Examples and quantities are illustrative.
What this choice protects
What it costs
When it fits
Compare the arrangements
Use cutoff 2
Flag every observation with score at least 2.
- What it protects
- Both toy signal-present cases are caught.
- What it costs
- N3 also triggers an alert.
- When it fits
- Fits a declared policy that accepts this false alert to avoid the illustrated miss.
Illustration note: The tiny labeled set exposes membership changes, not an estimated ROC.
Use cutoff 4
Require score at least 4 for an alert.
- What it protects
- The toy false alert disappears.
- What it costs
- S2 is now missed.
- When it fits
- Fits a declared policy that accepts the missed case to avoid the illustrated false alert.
Illustration note: Changing policy leaves the evidence scores and true labels fixed.
What this illustration does—and does not—establish
The source supplies the stated tension; the selected arrangements are bounded editorial illustrations. Costs and conditions remain part of the comparison.
- These four observations are an invented finite example, not estimates of real sensitivity, error rates, costs or base rates.
- No universal cutoff follows. A different detector can change the attainable tradeoff; shifting this cutoff cannot.
- This is one decision threshold, not a stateful hysteresis band or a calibrated probability forecast.
Source entries
Signal Detection Theory
This source passage supplies the contextual tension. The concrete arrangements and schematic examples are editorial illustrations, not measured findings.
Sensitivity versus Criterion (scopal)
T1 — Sensitivity versus Criterion (scopal). The framework's whole leverage is the orthogonality of sensitivity (the achievable trade-off) and criterion (where you sit on it), but the two are constantly conflated in practice — a high false-alarm rate is read as a bad instrument when it is a lenient cutoff. Failure mode: rebuilding the detector (expensive) when only the threshold needed moving, or endlessly tuning the threshold when the ROC itself is too poor to meet both targets. Diagnostic: would the complaint be fixed by moving the cutoff (criterion) or does it require a *better ROC* (sensitivity)? You cannot lower both error rates by moving the cutoff.
The source operation
In any setting where an observer must decide whether a particular state of the world — the "signal" — is present against a noisy background, every decision factorizes into two independent components. The first is *sensitivity*: how well the observer's internal evidence separates signal-present from signal-absent worlds. The second is a *criterion*: how much evidence the observer requires before responding "present." The choice of criterion is a free policy variable. It shifts the trade among hits, misses, false alarms, and correct rejections, moving the operating point along a receiver-operating-characteristic (ROC) curve whose *shape* is fixed by sensitivity. Confusing the two — reading high false-alarm rates as low sensitivity, or vice versa — produces persistent diagnostic errors.