Adam: A Method for Stochastic Optimization.¶
Kingma, D. P., & Ba, J. (2015). Adam: A Method for Stochastic Optimization. Proceedings of the 3rd International Conference on Learning Representations (ICLR).
Cited by¶
3 citations across 3 artifacts.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Gain Control
- A structurally identical applied instance is adaptive optimisers in machine learning, where per-parameter gradients are divided by a running estimate of their variance, gain-controlling each parameter update.
This sourceThe Adam optimizer divides each parameter's gradient by a running estimate of its (second-moment) variance — per-parameter gain control on gradient updates.
- A structurally identical applied instance is adaptive optimisers in machine learning, where per-parameter gradients are divided by a running estimate of their variance, gain-controlling each parameter update.
- Higher Order Function
- A machine-learning training pipeline runs the same structure mechanically: the first-order layer is a learning algorithm mapping datasets to models;
optimizer = Adam(beta1=0.9)is a higher-order function — a factory returning a configured update rule — and hyperparameter optimization is a higher-order rule taking learning algorithms as input and returning tuned ones.This sourceIn Proceedings of the 3rd International Conference on Learning Representations (ICLR). 2015. Defines the Adam optimizer; a configured optimizer is a factory returning a parameter-update rule, the higher-order layer over the learning algorithm.
- A machine-learning training pipeline runs the same structure mechanically: the first-order layer is a learning algorithm mapping datasets to models;
- Metaplasticity
- The ML parallel is exact and was a documented two-way port: an adaptive optimizer (Adam, learning-rate warmup-and-decay) maintains per-parameter learning rates that themselves update on the history of gradients — a slow controller wrapped around the fast weight-update rule, raising or lowering how much each weight moves per step based on accumulated gradient statistics.
This sourceAdaptive optimizer maintaining per-parameter learning rates that update on accumulated first- and second-moment gradient statistics—a slow controller over the fast weight-update rule.
- The ML parallel is exact and was a documented two-way port: an adaptive optimizer (Adam, learning-rate warmup-and-decay) maintains per-parameter learning rates that themselves update on the history of gradients — a slow controller wrapped around the fast weight-update rule, raising or lowering how much each weight moves per step based on accumulated gradient statistics.
Verification¶
Does it exist? Not checked yet. This entry carries no identifier to resolve. It was extracted from the citation as written in the article, normalized, and deduplicated against the rest of the registry.
Does it back the claim? Not recorded. Neither this nor any other of the 3 citations of this work carries a recorded support check.
Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.
See how references were verified.
Registry ID ref:671e2d495ef9 · see in the full table