Variable Generalization Performance of a Deep Learning Model to Detect Pneumonia in Chest Radiographs¶
Zech, J. R., Badgeley, M. A., Liu, M., Costa, A. B., Titano, J. J., & Oermann, E. K. (2018). Variable Generalization Performance of a Deep Learning Model to Detect Pneumonia in Chest Radiographs: A Cross-Sectional Study. PLOS Medicine, 15(11).
Cited by¶
4 citations across 4 artifacts.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Cue Outcome Decoupling
- Make it concrete: a pneumonia detector trained at one hospital latches onto a scanner watermark — a token burned into images by that site's portable X-ray machine, which was used disproportionately on already-sick inpatients, so the watermark correlated with disease in training.
This sourceDocuments a pneumonia detector keying on a scanner/hospital signal correlated with disease prevalence, failing across sites.
- Make it concrete: a pneumonia detector trained at one hospital latches onto a scanner watermark — a token burned into images by that site's portable X-ray machine, which was used disproportionately on already-sick inpatients, so the watermark correlated with disease in training.
- Out Of Distribution Detection
- A single worked instance shows the move: a chest-X-ray classifier trained on adults, deployed at a paediatric clinic, confidently mis-diagnoses paediatric anatomy until a retrofit OOD detector flags those inputs as out-of-distribution and routes them to a human radiologist — in-distribution performance unchanged, the silent failure converted into an explicit referral.
This sourceShows chest-radiograph CNNs exploit site-specific confounders and generalize poorly to new hospitals/populations, so on-site performance overstates real-world reliability — empirical grounding for distribution shift degrading clinical imaging models and motivating caution/human review on shifted inputs.
- A single worked instance shows the move: a chest-X-ray classifier trained on adults, deployed at a paediatric clinic, confidently mis-diagnoses paediatric anatomy until a retrofit OOD detector flags those inputs as out-of-distribution and routes them to a human radiologist — in-distribution performance unchanged, the silent failure converted into an explicit referral.
- Relevance Substitution
- During training the model discovers a substituted channel: a metal token or scanner artefact that hospitals with sicker populations happen to stamp on their films correlates with the label, so the model's inference runs through the artefact.
This sourceShows a pneumonia classifier keying on scanner/hospital artefacts rather than lung pathology, collapsing on out-of-distribution equipment.
- During training the model discovers a substituted channel: a metal token or scanner artefact that hospitals with sicker populations happen to stamp on their films correlates with the label, so the model's inference runs through the artefact.
- Shortcut Learning
- The identical structure governs medical imaging: a pneumonia detector trained across hospitals latches onto the X-ray machine's metadata or image-edge tokens that correlate with the sicker hospital's higher base rate, rather than the lung pathology; it collapses at a new hospital, and the fix is multi-site training that varies the scanner independently of the diagnosis.
This sourceShows a pneumonia detector learning hospital/scanner-specific artifacts that correlate with disease prevalence, collapsing at a new site — the medical-imaging shortcut.
- The identical structure governs medical imaging: a pneumonia detector trained across hospitals latches onto the X-ray machine's metadata or image-edge tokens that correlate with the sicker hospital's higher base rate, rather than the lung pathology; it collapses at a new hospital, and the fix is multi-site training that varies the scanner independently of the diagnosis.
Verification¶
This reference passed the adversarial substantiation pipeline: it was checked to exist and to support the claim it is attached to. See how references were verified.
Registry ID ref:56f02acd9ad6 · see in the full table