Dynamic Programming and Markov Processes¶
Howard, R. A. (1960). Dynamic Programming and Markov Processes. Technology Press.
Cited by¶
3 citations across 3 artifacts.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Dynamic Programming
- The method presumes (a) a problem with optimal substructure — the optimal solution to the whole can be expressed in terms of optimal solutions to subproblems of the same form; (b) overlapping subproblems — naive recursion would re-solve the same subproblems exponentially many times; © a well-defined state representation that captures all information needed to characterize a subproblem; and (d) a recurrence relation that expresses the optimal value at one state in terms of optimal values at related states.
This sourceEstablishes policy iteration (value-determination + policy-improvement) as a scalable exact-solution algorithm for finite-state Markov decision processes.
- The method presumes (a) a problem with optimal substructure — the optimal solution to the whole can be expressed in terms of optimal solutions to subproblems of the same form; (b) overlapping subproblems — naive recursion would re-solve the same subproblems exponentially many times; © a well-defined state representation that captures all information needed to characterize a subproblem; and (d) a recurrence relation that expresses the optimal value at one state in terms of optimal values at related states.
- Markov Decision Processes (MDPs)
- Howard (1960), in Dynamic Programming and Markov Processes, systematized policy iteration.
This sourceSystematized policy iteration as a solution method for MDPs and developed the modern algorithmic vocabulary for value functions and policy improvement steps.
- Howard (1960), in Dynamic Programming and Markov Processes, systematized policy iteration.
Mechanisms¶
- Policy Iteration
- Its defining property is that improvement is guaranteed monotone
This sourceDescribes policy improvement as guaranteed monotone while each policy change requires an expensive full reevaluation.
- Its defining property is that improvement is guaranteed monotone
Verification¶
Does it exist? Not checked yet. This entry carries no identifier to resolve. It was extracted from the citation as written in the article, normalized, and deduplicated against the rest of the registry.
Does it back the claim? Not recorded. Neither this nor any other of the 3 citations of this work carries a recorded support check.
Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.
See how references were verified.
Links previously used in the corpus¶
Before the registry existed this work was also linked 1 other way.
Registry ID ref:179cb5b31336 · see in the full table