On Calibration of Modern Neural Networks.¶
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017). On Calibration of Modern Neural Networks. Proceedings of the 34th International Conference on Machine Learning, 1321-1330.
Cited by¶
5 citations across 5 artifacts.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Calibration
- In machine learning, recognizing that a model can be accurate without being calibrated opens new strategies, as Guo, Pleiss, Sun, and Weinberger (2017) demonstrated for modern deep neural networks.
This sourceDemonstrates that modern deep networks are accurate but poorly calibrated and that temperature scaling (a single-parameter Platt variant) restores calibration post-hoc without retraining.
- In machine learning, recognizing that a model can be accurate without being calibrated opens new strategies, as Guo, Pleiss, Sun, and Weinberger (2017) demonstrated for modern deep neural networks.
- Confidence Annotation
- In machine learning it is calibrated confidence scores and ensemble disagreement that feed abstention thresholds and human-in-the-loop routing.
This sourceShows modern classifiers are miscalibrated and corrects them (temperature scaling) so confidence scores match accuracy; supports calibrated-score and abstention-routing claims.
- In machine learning it is calibrated confidence scores and ensemble disagreement that feed abstention thresholds and human-in-the-loop routing.
- Out Of Distribution Detection
- Calibration is computed on in-distribution cases and is blind to novel ones — a perfectly calibrated model still answers confidently and wrongly out of scope.
This sourceShows modern networks are systematically miscalibrated and that calibration methods (e.g., temperature scaling) are tuned on in-distribution validation data; supports the point that calibration fixes confidence within the training distribution, not out of scope.
- Calibration is computed on in-distribution cases and is blind to novel ones — a perfectly calibrated model still answers confidently and wrongly out of scope.
Mechanisms¶
- Calibration Curve Review
- The discipline is to treat a miscalibration finding as a mandate to fix the score,
This sourceShows that poorly calibrated neural-network confidence scores can be improved with post-processing calibration such as temperature scaling.
- The discipline is to treat a miscalibration finding as a mandate to fix the score,
- Confidence Threshold Router
- Its central failure mode is miscalibration — if the model is over-confident, its 0.98 is not 98% likely to be right, and every threshold built on that number routes badly; worse, distribution shift silently degrades calibration while the cutoffs stay put.
This sourceShows that modern neural networks can be overconfident and poorly calibrated, so a reported confidence need not match the empirical probability of correctness.
- Its central failure mode is miscalibration — if the model is over-confident, its 0.98 is not 98% likely to be right, and every threshold built on that number routes badly; worse, distribution shift silently degrades calibration while the cutoffs stay put.
Verification¶
Does it exist? Not checked yet. This entry carries no identifier to resolve. It was extracted from the citation as written in the article, normalized, and deduplicated against the rest of the registry.
Does it back the claim? Not recorded. Neither this nor any other of the 5 citations of this work carries a recorded support check.
Was it audited? Yes. A second, independent pass read the citation against the article text and recorded a verdict.
Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.
See how references were verified.
Registry ID ref:5e36cc8392df · see in the full table