Verification of Forecasts Expressed in Terms of Probability.¶
Brier, G. W. (1950). Verification of Forecasts Expressed in Terms of Probability. Monthly Weather Review, 78(1), 1-3.
Cited by¶
14 citations across 14 artifacts.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Calibration
- A primary function of calibration is to distinguish between bias (systematic offset that can be corrected) and noise (random fluctuation that cannot), a separation Brier (1950) made operational through his probability-score framework that decomposes verification error into systematic and stochastic components.
This sourceIntroduces the Brier score, the operational quadratic scoring rule for probability-of-binary-outcome forecasts on which later calibration decompositions are built.
- A primary function of calibration is to distinguish between bias (systematic offset that can be corrected) and noise (random fluctuation that cannot), a separation Brier (1950) made operational through his probability-score framework that decomposes verification error into systematic and stochastic components.
- Confidence Annotation
- A reliability diagram plots stated probability against observed frequency, and a proper scoring rule — the Brier score \(\frac{1}{N}\sum (p_i - o_i)^2\), where \(o_i \in \{0,1\}\) is the outcome — is minimized in expectation only by honest, calibrated probabilities, so it both measures miscalibration and incentivizes against it.
This sourceIntroduces the Brier score, a proper scoring rule minimized in expectation by honest, calibrated probabilities.
- A reliability diagram plots stated probability against observed frequency, and a proper scoring rule — the Brier score \(\frac{1}{N}\sum (p_i - o_i)^2\), where \(o_i \in \{0,1\}\) is the outcome — is minimized in expectation only by honest, calibrated probabilities, so it both measures miscalibration and incentivizes against it.
- Uncertainty
- Calibration is measurable
This sourceOriginal Brier score paper providing the operational scoring rule whose decomposition cleanly separates systematic bias (correctable through calibration) from irreducible stochastic noise.
- Calibration is measurable
Domain-specific¶
Mechanisms¶
- Calibrated Probability Elicitation
- Score calibration. Compare stated confidence to actual accuracy — a proper scoring rule such as the Brier score
This sourceIntroduces a probability score that compares stated confidence with observed accuracy and rewards appropriately calibrated uncertainty.
- Score calibration. Compare stated confidence to actual accuracy — a proper scoring rule such as the Brier score
- Calibration Exercise
- The exercise scores accuracy with a proper rule
This sourceIntroduces a squared-error score for evaluating forecasts stated as probabilities.
- The exercise scores accuracy with a proper rule
- Confidence Scope Update Memo
- The discipline it enforces is calibration: a confidence claim should match the evidence's actual accuracy
This sourceScores probabilistic confidence by its squared deviation from observed outcomes, penalizing probabilities that are too high or too low.
- The discipline it enforces is calibration: a confidence claim should match the evidence's actual accuracy
- Forecast Backtesting Cadence
- They log both errors with their directions, then compute a calibration score across all their past horizon forecasts and find a systematic pattern: the team runs roughly 30% overconfident at the short horizon and keeps its long-horizon bands too narrow.
This sourceDefines an aggregate mean-square score for evaluating probabilistic forecasts against observed outcomes.
- They log both errors with their directions, then compute a calibration score across all their past horizon forecasts and find a systematic pattern: the team runs roughly 30% overconfident at the short horizon and keeps its long-horizon bands too narrow.
- Forecast Backtesting Review
- Tracking calibration across many forecasts — the discipline behind scoring rules like the Brier score — is what separates a genuinely improving forecaster from a lucky one.
This sourceIntroduces the probability-forecast scoring rule now known as the Brier score.
- Tracking calibration across many forecasts — the discipline behind scoring rules like the Brier score — is what separates a genuinely improving forecaster from a lucky one.
- Forecast Update Trigger Log
- Scoring rule — whether resolved items are scored (e.g. by a Brier score) to feed calibration back into future triggers.
This sourceIntroduces a quantitative score for verifying forecasts expressed in terms of probability.
- Scoring rule — whether resolved items are scored (e.g. by a Brier score) to feed calibration back into future triggers.
- Post-Decision Calibration Review
- If it rained on only 55% of them, the office's 70% is systematically overconfident, and the review says so with a calibration curve and a Brier score
This sourceIntroduces a quadratic score comparing probability forecasts with realized categorical outcomes; it does not introduce a calibration curve or isolate the 70%-versus-55% calibration gap.
- If it rained on only 55% of them, the office's 70% is systematically overconfident, and the review says so with a calibration curve and a Brier score
- Posterior Risk Estimation
- Its strength is that it produces a comparable, forward-looking risk by which many cases can be ranked and governed, and its calibration discipline
This sourceIntroduces a score for verifying probability forecasts by comparing stated probabilities with observed outcomes.
- Its strength is that it produces a comparable, forward-looking risk by which many cases can be ranked and governed, and its calibration discipline
- Probabilistic Forecast
- Over thousands of past "60%" forecasts, the service checks whether rain actually fell on roughly sixty percent of those days, summarizing the gap with a Brier score and correcting any systematic over- or under-confidence.
This sourceDefines a quadratic score that summarizes the error between forecast probabilities and realized outcomes.
- Over thousands of past "60%" forecasts, the service checks whether rain actually fell on roughly sixty percent of those days, summarizing the gap with a Brier score and correcting any systematic over- or under-confidence.
- Probability Estimate
- A Brier score aggregates this over all the day's forecasts into a single number that rewards being both accurate and calibrated.
This sourceDefines the mean squared probability score used to summarize and verify repeated probabilistic forecasts, including their reliability.
- A Brier score aggregates this over all the day's forecasts into a single number that rewards being both accurate and calibrated.
Verification¶
Does it exist? Not confirmed. The DOI recorded for this work did not resolve when it was last checked, so it is shown above but not linked. The citation may still be sound; the recorded identifier is not. Flagged for repair.
Does it back the claim? Not recorded. Neither this nor any other of the 14 citations of this work carries a recorded support check.
Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.
See how references were verified.
Links previously used in the corpus¶
Before the registry existed this work was also linked 3 other ways.
- https://doi.org/10.1175/1520-0493%281950%29078%3C0001%3AVOFEIT%3E2.0.CO%3B2 ×3
- https://doi.org/10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2 ×2
- https://journals.ametsoc.org/view/journals/mwre/78/1/1520-0493_1950_078_0001_vofeit_2_0_co_2.xml ×1
Registry ID ref:c932fba111c0 · see in the full table