A Stochastic Approximation Method¶
Robbins, H., & Monro, S. (1951). A Stochastic Approximation Method. The Annals of Mathematical Statistics, 22(3), 400-407.
Cited by¶
3 citations across 3 artifacts.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Gradient
- In optimization and machine learning, gradient descent and its stochastic variants (originating with Cauchy and revived for large-scale ML by Robbins-Monro
This sourceFoundational paper on stochastic approximation: establishes the algorithmic framework underlying stochastic gradient descent and stochastic optimization in modern machine learning.
- In optimization and machine learning, gradient descent and its stochastic variants (originating with Cauchy and revived for large-scale ML by Robbins-Monro
- Stochasticity vs. Determinism
- As Robbins and Monro (1951) established in their foundational stochastic approximation method, the training process is stochastic, the final model is deterministic, and the deployment performance is stochastic.
This sourceFoundational paper on stochastic approximation: establishes the algorithmic framework underlying stochastic gradient descent and stochastic optimization in modern machine learning.
- As Robbins and Monro (1951) established in their foundational stochastic approximation method, the training process is stochastic, the final model is deterministic, and the deployment performance is stochastic.
Mechanisms¶
- Step-Size Throttle
- This is the same logic as a learning-rate schedule, and its convergence conditions are the classic ones from stochastic approximation
This sourceEstablishes convergence conditions for a stochastic-approximation update governed by a prescribed sequence of positive step sizes.
- This is the same logic as a learning-rate schedule, and its convergence conditions are the classic ones from stochastic approximation
Verification¶
This reference passed the adversarial substantiation pipeline: it was checked to exist and to support the claim it is attached to. See how references were verified.
Registry ID ref:4fe0feda6f24 · see in the full table