The Clinician and Dataset Shift in Artificial Intelligence¶
Finlayson, S. G., Subbaswamy, A., Singh, K., Bowers, J., Kupke, A., Zittrain, J., Kohane, I. S., et al. (2021). The Clinician and Dataset Shift in Artificial Intelligence. New England Journal of Medicine, 385(3), 283-286.
Cited by¶
2 citations across 2 artifacts.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Data Drift
- In clinical medicine, diagnostic models, dosing rules, and guidelines calibrated on one patient population drift as patient mix, comorbidity profiles, and treatment contexts evolve.
This sourceDescribes how clinical models and guidelines drift as patient mix, comorbidity profiles, and treatment contexts evolve.
Supported in partVerified against the work's full text
Finlayson et al. show clinical ML models degrade as patient populations, treatments and nomenclature change, but do not address dosing rules or guidelines drifting as such.
“Dataset shift occurs when a machine-learning system underperforms because of a mismatch between the data set with which it was developed and the data on which it is deployed.”
- In clinical medicine, diagnostic models, dosing rules, and guidelines calibrated on one patient population drift as patient mix, comorbidity profiles, and treatment contexts evolve.
Domain-specific¶
- Gold-Standard Erosion
- A clinical prediction model trained and validated against 2015 pathology-confirmed diagnoses scores 0.89 AUC against the 2015 label; when the radiology society updates diagnostic criteria in 2024, re-evaluation on a fresh cohort labeled to the new criterion yields 0.78 AUC, and the naive interpretation is model degradation.
This sourceOutlines how to identify and potentially mitigate common sources of dataset shift in clinical machine-learning systems. Identifies dataset shift in clinical machine-learning systems as a source of apparent model-performance change.
Supported in partVerified against the publisher's abstract
“This letter outlines how to identify, and potentially mitigate, common sources of “dataset shift” in machine-learning systems.”
- A clinical prediction model trained and validated against 2015 pathology-confirmed diagnoses scores 0.89 AUC against the 2015 label; when the radiology society updates diagnostic criteria in 2024, re-evaluation on a fresh cohort labeled to the new criterion yields 0.78 AUC, and the naive interpretation is model degradation.
Verification¶
Does it exist? Confirmed. This work's DOI resolves to a registered record, which fixes its identity. That is all it fixes.
Does it back the claim? Read against the text for 2 of 2 citations: 2 supported in part. Each verdict is shown under its citation below, with what in the work backs the sentence.
Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.
See how references were verified.
Registry ID ref:564484cf5f3c · see in the full table