Overcoming Catastrophic Forgetting in Neural Networks.¶
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., et al. (2017). Overcoming Catastrophic Forgetting in Neural Networks. Proceedings of the National Academy of Sciences, 114(13), 3521-3526.
Cited by¶
4 citations across 4 artifacts.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Cognitive Flexibility
- In machine learning it shows up as meta-learning, mixture-of-experts gating, and continual learning, with catastrophic forgetting as the pathological loss of the repertoire.
This sourceCharacterizes catastrophic forgetting as loss of a learned repertoire and proposes elastic weight consolidation to preserve it.
- In machine learning it shows up as meta-learning, mixture-of-experts gating, and continual learning, with catastrophic forgetting as the pathological loss of the repertoire.
- Memory Consolidation
- The diagnostic payoff is concrete — continual-learning methods (elastic weight consolidation) treat "catastrophic forgetting" not as a learning failure but as a consolidation failure (the window was disrupted by un-interleaved new tasks) and intervene in the stabilization stage, not in encoding.
This sourceIntroduces Elastic Weight Consolidation, engineering a consolidation stage that protects weights important to prior tasks.
- The diagnostic payoff is concrete — continual-learning methods (elastic weight consolidation) treat "catastrophic forgetting" not as a learning failure but as a consolidation failure (the window was disrupted by un-interleaved new tasks) and intervene in the stabilization stage, not in encoding.
- Two-Store Architecture
- In machine learning, replay buffers and online weights pair with target networks and consolidated weights, as in DQN, elastic weight consolidation, and Dyna-style designs; without the slow store, online updates produce catastrophic forgetting.
This sourceIntroduces elastic weight consolidation; demonstrates catastrophic forgetting absent a consolidation/slow-store mechanism.
- In machine learning, replay buffers and online weights pair with target networks and consolidated weights, as in DQN, elastic weight consolidation, and Dyna-style designs; without the slow store, online updates produce catastrophic forgetting.
Mechanisms¶
- Selective Parameter Freezing
- Its strength is that it lets several context-maps coexist on one substrate without keeping a full copy per context, directly countering catastrophic forgetting
This sourceShows that elastic weight consolidation can retain several sequentially learned tasks in one network by selectively protecting weights important to earlier tasks.
- Its strength is that it lets several context-maps coexist on one substrate without keeping a full copy per context, directly countering catastrophic forgetting
Verification¶
This reference passed the adversarial substantiation pipeline: it was checked to exist and to support the claim it is attached to. See how references were verified.
Registry ID ref:849082368a44 · see in the full table