Learning to Predict by the Methods of Temporal Differences.¶
Sutton, R. S. (1988). Learning to Predict by the Methods of Temporal Differences. Machine Learning, 3(1), 9-44.
Cited by¶
2 citations across 2 artifacts.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Markov Decision Processes (MDPs)
- The 1990s transformation was the convergence with reinforcement learning: Watkins and Dayan (1992) on Q-learning convergence (following Watkins' 1989 thesis), Sutton's
This sourceFoundational paper on temporal-difference (TD) learning, the algorithmic core of model-free reinforcement learning for MDPs.
- The 1990s transformation was the convergence with reinforcement learning: Watkins and Dayan (1992) on Q-learning convergence (following Watkins' 1989 thesis), Sutton's
- Preparatory Field Conditioning
- The arrows run one way and accountability runs the other: effect flows forward through the state, while judgement of the conditioning act stops at the state, because the act has no result of its own.
This sourceIt is adverse on direction — in temporal-difference learning credit propagates backward, from later states to earlier ones — which is why the entry's earlier wording that "value flows forward through the state" has been corrected to "effect"; the accountability asymmetry itself is this entry's own formulation, not a result in this paper.
Supported in partVerified against the publisher's abstract
Sutton's abstract supports the credit-assignment half: earlier predictions are judged against the next prediction; it does not address forward effect flow or acts with no result of their own.
“the new methods assign credit by means of the difference between temporally successive predictions”
- The arrows run one way and accountability runs the other: effect flows forward through the state, while judgement of the conditioning act stops at the state, because the act has no result of its own.
Verification¶
Does it exist? Confirmed. This work's DOI resolves to a registered record, which fixes its identity. That is all it fixes.
Does it back the claim? Read against the text for 1 of 2 citations: 1 supported in part. Each verdict is shown under its citation below, with what in the work backs the sentence.
Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.
See how references were verified.
Registry ID ref:53d565ec606e · see in the full table