Shortcut Learning in Deep Neural Networks.¶
Geirhos, R., Jacobsen, J., Michaelis, C., Zemel, R., Brendel, W., Bethge, M., & Wichmann, F. A. (2020). Shortcut Learning in Deep Neural Networks. Nature Machine Intelligence, 2, 665-673.
Cited by¶
7 citations across 7 artifacts.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Cue Outcome Decoupling
- In machine learning under distribution shift, a classifier latches onto a scanner watermark correlated with disease at the training site but not the new site — shortcut learning and spurious correlation.
This sourceDefines shortcut learning: models latch onto spurious correlations that fail under distribution shift, with better-fit shortcuts generalizing worse.
- In machine learning under distribution shift, a classifier latches onto a scanner watermark correlated with disease at the training site but not the new site — shortcut learning and spurious correlation.
- Feature Engineering
- Running the other way, education psychometrics' long experience with construct validity — does this feature actually measure the latent thing it claims to? — ports into machine learning, where its absence now appears under names like spurious correlation and shortcut learning.
This sourceNames spurious-correlation / shortcut learning — the ML manifestation of features lacking construct validity — supporting the construct-validity transfer into ML.
- Running the other way, education psychometrics' long experience with construct validity — does this feature actually measure the latent thing it claims to? — ports into machine learning, where its absence now appears under names like spurious correlation and shortcut learning.
- Proxy-Target Divergence
- The same pattern recurs in a hedge whose historical spread relationships decouple from default risk during a crisis (stress decoupling), a sea-turtle hatchery where a brightest-horizon cue decouples from ocean direction under light pollution (environmental decoupling), and a pneumonia classifier whose scanner artefact decouples from pneumonia signal under deployment shift (distribution-shift decoupling).
This sourceShows classifiers latching onto spurious features (e.g., scanner artifacts) that decouple from the intended signal under distribution shift.
- The same pattern recurs in a hedge whose historical spread relationships decouple from default risk during a crisis (stress decoupling), a sea-turtle hatchery where a brightest-horizon cue decouples from ocean direction under light pollution (environmental decoupling), and a pneumonia classifier whose scanner artefact decouples from pneumonia signal under deployment shift (distribution-shift decoupling).
- Relevance Substitution
- In machine learning, shortcut learning substitutes spurious features (image background, scanner artefact, watermark) for the task-defining features, the model's inference running through the irrelevant channel.
This sourceNames and surveys shortcut learning, where models rely on spurious features rather than task-defining ones.
- In machine learning, shortcut learning substitutes spurious features (image background, scanner artefact, watermark) for the task-defining features, the model's inference running through the irrelevant channel.
- Shortcut Learning
- The general claim is compact: optimization under a sufficient statistic finds the cheapest sufficient statistic, not the target.
This sourceCanonical statement of shortcut learning: optimizers seize cheap predictive features (the 'cheapest sufficient statistic') that succeed in-distribution and fail under distribution shift; surveys ML cases and stress-set/OOD diagnostics.
- The general claim is compact: optimization under a sufficient statistic finds the cheapest sufficient statistic, not the target.
Mechanisms¶
- Convolutional Feature Extractor
- Its failure mode is that a learned kernel will happily latch onto whatever correlates with the label in the training data rather than what actually causes it — shortcut learning.
This sourceDefines shortcut learning as a model's reliance on unintended decision rules that perform on standard data but fail under changed conditions.
- Its failure mode is that a learned kernel will happily latch onto whatever correlates with the label in the training data rather than what actually causes it — shortcut learning.
- Supervised Representation Learning
- Its failure mode is shortcut learning: the model latches onto a nuisance feature that happens to correlate with the label in the training set, so its output looks clean and then collapses under distribution shift.
This sourceDefines shortcut learning as reliance on unintended training-set correlations that can fail under distribution shift.
- Its failure mode is shortcut learning: the model latches onto a nuisance feature that happens to correlate with the label in the training set, so its output looks clean and then collapses under distribution shift.
Verification¶
This reference passed the adversarial substantiation pipeline: it was checked to exist and to support the claim it is attached to. See how references were verified.
Registry ID ref:ce3d33590cf8 · see in the full table