Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping

Ng, A. Y., Harada, D., & Russell, S. (1999). Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping: Theory and Application to Reward Shaping. Proceedings of the Sixteenth International Conference on Machine Learning, 278-287.

Type

Conference paper

Intellectual base

Primary research

Year

1999

Pages

278-287

DOI

10.5555/645528.657613 did not resolve

Link

https://ai.stanford.edu/~ang/papers/shaping-icml99.pdf

Cited from

behavioral_economics

Cited by

2 citations across 2 artifacts.

Each citation links to the sentence it supports in the citing article.

Primes

• Reference-Point Dependence
  • The subtraction can leave the optimum untouched while transforming the learning dynamics — reference dependence in a system with no experience of gain or loss whatsoever.

    This sourceProves that potential-based shaping leaves the optimal policy unchanged while altering the learning dynamics.

Mechanisms

• Almost-Reward Annotation
  • The literature on potential-based reward shaping is precise about this danger — a shaping signal is safe only when it cannot change which behavior is optimal.

    This sourceEstablishes potential-based reward shaping as the general policy-invariant form and shows that other shaping transformations can change the optimal policy.

Verification

Does it exist? Not confirmed. The DOI recorded for this work did not resolve when it was last checked, so it is shown above but not linked. The citation may still be sound; the recorded identifier is not. Flagged for repair.

Does it back the claim? Not recorded. Neither this nor any other of the 2 citations of this work carries a recorded support check.

Was it audited? Yes. A second, independent pass read the citation against the article text and recorded a verdict.

Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.

See how references were verified.

Links previously used in the corpus

Before the registry existed this work was also linked 1 other way.

• https://dl.acm.org/doi/10.5555/645528.657613 ×1

Registry ID ref:53210b5e3465 · see in the full table