Learning to Predict by the Methods of Temporal Differences.

Sutton, R. S. (1988). Learning to Predict by the Methods of Temporal Differences. Machine Learning, 3(1), 9-44.

Type

Journal article

Intellectual base

Primary research

Year

1988

DOI

10.1007/bf00115009

Link

https://doi.org/10.1007/BF00115009

Cited from

military_security, operations_research

Cited by

2 citations across 2 artifacts.

Each citation links to the sentence it supports in the citing article.

Primes

• Markov Decision Processes (MDPs)
  • … (1963), with substantial development through the 1960s-1980s in applications (inventory management, maintenance, queueing control, game theory). The 1990s transformation was the convergence with reinforcement learning: Watkins and Dayan (1992) on Q-learning convergence (following Watkins' 1989 thesis), Sutton's

    This sourceFoundational paper on temporal-difference (TD) learning, the algorithmic core of model-free reinforcement learning for MDPs.

• Preparatory Field Conditioning
  • The arrows run one way and accountability runs the other: effect flows forward through the state, while judgement of the conditioning act stops at the state, because the act has no result of its own.

    This sourceIt is adverse on direction — in temporal-difference learning credit propagates backward, from later states to earlier ones — which is why the entry's earlier wording that "value flows forward through the state" has been corrected to "effect"; the accountability asymmetry itself is this entry's own formulation, not a result in this paper.

Verification

This reference passed the adversarial substantiation pipeline: it was checked to exist and to support the claim it is attached to. See how references were verified.

Registry ID ref:53d565ec606e · see in the full table