Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imaging.¶
Oakden-Rayner, L., Dunnmon, J., Carneiro, G., & Re, C. (2020). Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imaging. Proceedings of the ACM Conference on Health, Inference, and Learning, 151-159.
Cited by¶
1 citation across 1 artifact.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Ground Truth
- But the prime's warning about the reference's own error structure is concrete and consequential: if the "ground truth" for a chest-X-ray model is radiologist consensus rather than confirmed disease status, the model is trained and scored to reproduce radiologists' judgments, including their systematic misses
This sourceShows medical-imaging models scored against radiologist-consensus labels reproduce systematic reader errors rather than true disease status.
Supported in partVerified against the work's full text
Oakden-Rayner et al. argue that labels from clinical practice have reduced quality for findings radiologists elide, which supports the general warning, but not radiologist-consensus ground truth for chest X-rays.
“We hypothesize that labels derived from clinical practice are likely to demonstrate this phenomenon; that irrelevant or unimportant findings are often elided by radiologists, leading to reduced label quality for less”
- But the prime's warning about the reference's own error structure is concrete and consequential: if the "ground truth" for a chest-X-ray model is radiologist consensus rather than confirmed disease status, the model is trained and scored to reproduce radiologists' judgments, including their systematic misses
Verification¶
Does it exist? Confirmed. This work's DOI resolves to a registered record, which fixes its identity. That is all it fixes.
Does it back the claim? Read against the text for 1 of 1 citation: 1 supported in part. Each verdict is shown under its citation below, with what in the work backs the sentence.
Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.
See how references were verified.
Registry ID ref:ac7c5ade7450 · see in the full table