Prioritized Experience Replay.¶
Schaul, T., Quan, J., Antonoglou, I., & Silver, D. (2016). Prioritized Experience Replay. International Conference on Learning Representations (ICLR).
Cited by¶
3 citations across 3 artifacts.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Replay
- Neuroscience (origin) — hippocampal place-cell sequences from waking activity reappear time-compressed during slow-wave sleep and quiet wakefulness, causally implicated in memory consolidation and spatial planning. Reinforcement learning — experience-replay buffers store transitions during interaction and resample them off-policy during training; prioritized replay reweights by temporal-difference error, and replay is what stabilized deep RL.
This sourceReweights replay sampling by temporal-difference error under a bounded budget, with importance-sampling correction for the induced bias.
- Neuroscience (origin) — hippocampal place-cell sequences from waking activity reappear time-compressed during slow-wave sleep and quiet wakefulness, causally implicated in memory consolidation and spatial planning. Reinforcement learning — experience-replay buffers store transitions during interaction and resample them off-policy during training; prioritized replay reweights by temporal-difference error, and replay is what stabilized deep RL.
Mechanisms¶
- Prediction-Error Replay Buffer
- Where a live gate asks "does this residual matter now," the buffer asks "which residuals will teach us something later," and keeps those.
This sourceStores past transitions for later training and prioritizes high-error experiences for more frequent replay.
- Where a live gate asks "does this residual matter now," the buffer asks "which residuals will teach us something later," and keeps those.
- Prioritized Trace Sampling
- Its strength is concentration: it turns a fixed replay budget toward the traces that still teach, which is why it can sharply accelerate learning on tasks dominated by a few hard cases and rare events.
This sourceSchaul and colleagues show that prioritizing still-informative replay transitions can accelerate learning, especially when rare successes or hard cases are buried among redundant experience.
- Its strength is concentration: it turns a fixed replay budget toward the traces that still teach, which is why it can sharply accelerate learning on tasks dominated by a few hard cases and rare events.
Verification¶
Does it exist? Not checked yet. This entry carries no identifier to resolve. It was extracted from the citation as written in the article, normalized, and deduplicated against the rest of the registry.
Does it back the claim? Not recorded. Neither this nor any other of the 3 citations of this work carries a recorded support check.
Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.
See how references were verified.
Registry ID ref:6baa83257a6d · see in the full table