A Stochastic Approximation Method¶
Robbins, H., & Monro, S. (1951). A Stochastic Approximation Method. The Annals of Mathematical Statistics, 22(3), 400-407.
Cited by¶
3 citations across 3 artifacts.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Gradient
- In optimization and machine learning, gradient descent and its stochastic variants (originating with Cauchy and revived for large-scale ML by Robbins-Monro
This sourceFoundational paper on stochastic approximation: establishes the algorithmic framework underlying stochastic gradient descent and stochastic optimization in modern machine learning.
- In optimization and machine learning, gradient descent and its stochastic variants (originating with Cauchy and revived for large-scale ML by Robbins-Monro
- Stochasticity vs. Determinism
- As Robbins and Monro (1951) established in their foundational stochastic approximation method, the training process is stochastic, the final model is deterministic, and the deployment performance is stochastic.
This sourceFoundational paper on stochastic approximation: establishes the algorithmic framework underlying stochastic gradient descent and stochastic optimization in modern machine learning.
Supported in partVerified against the publisher's abstract
The 1951 abstract states a stochastic approximation method for solving M(x)=alpha; it says nothing about training runs, deterministic final models, or deployment performance.
“We give a method for making successive experiments at levels $x_1,x_2,\cdots$ in such a way that $x_n$ will tend to $\theta$ in probability.”
- As Robbins and Monro (1951) established in their foundational stochastic approximation method, the training process is stochastic, the final model is deterministic, and the deployment performance is stochastic.
Mechanisms¶
- Step-Size Throttle
- This is the same logic as a learning-rate schedule, and its convergence conditions are the classic ones from stochastic approximation
This sourceEstablishes convergence conditions for a stochastic-approximation update governed by a prescribed sequence of positive step sizes.
- This is the same logic as a learning-rate schedule, and its convergence conditions are the classic ones from stochastic approximation
Verification¶
Does it exist? Confirmed. This work's DOI resolves to a registered record, which fixes its identity. That is all it fixes.
Does it back the claim? Read against the text for 1 of 3 citations: 1 supported in part. Each verdict is shown under its citation below, with what in the work backs the sentence.
Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.
See how references were verified.
Registry ID ref:4fe0feda6f24 · see in the full table