Dynamic Programming and Markov Processes

Howard, R. A. (1960). Dynamic Programming and Markov Processes. MIT Press.

Type

Book

Intellectual base

Review or monograph

Year

1960

Link

https://archive.org/details/dynamicprogrammi0000howamatched

Cited from

operations_research

Cited by

2 citations across 2 artifacts.

Each citation links to the sentence it supports in the citing article.

Primes

• Dynamic Programming
  • The method presumes (a) a problem with optimal substructure — the optimal solution to the whole can be expressed in terms of optimal solutions to subproblems of the same form; (b) overlapping subproblems — naive recursion would re-solve the same subproblems exponentially many times; © a well-defined state representation that captures all information needed to characterize a subproblem; and (d) a recurrence relation that expresses the optimal value at one state in terms of optimal values at related states.

    This sourceEstablishes policy iteration (value-determination + policy-improvement) as a scalable exact-solution algorithm for finite-state Markov decision processes

• Markov Decision Processes (MDPs)
  • Markov Decision Processes were formalized by Bellman (1957) in Dynamic Programming, with the Bellman equation and the principle of optimality as foundational results.

    This sourceSystematized policy iteration as a solution method for MDPs and developed the modern algorithmic vocabulary for value functions and policy improvement steps.

Verification

This reference passed the adversarial substantiation pipeline: it was checked to exist and to support the claim it is attached to. See how references were verified.

Links previously used in the corpus

Before the registry existed this work was also linked 1 other way.

• https://www.google.com/books/edition/Dynamic_Programming_and_Markov_Processes/0N3-GAAACAAJ ×1

Registry ID ref:179cb5b31336 · see in the full table