SGDR: Stochastic Gradient Descent with Warm Restarts.¶
Loshchilov, I., & Hutter, F. (2017). SGDR: Stochastic Gradient Descent with Warm Restarts. International Conference on Learning Representations (ICLR).
Cited by¶
1 citation across 1 artifact.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Annealing
- Machine learning: learning-rate schedules (cosine annealing, warm restarts, polynomial decay) and the noise schedules of diffusion-model training are explicit anneal-down protocols.
This sourceIntroduces cosine annealing of the learning rate with warm restarts, an explicit anneal-down schedule for deep-network training in which the schedule, not the peak rate, governs the quality of the minimum reached.
- Machine learning: learning-rate schedules (cosine annealing, warm restarts, polynomial decay) and the noise schedules of diffusion-model training are explicit anneal-down protocols.
Verification¶
This reference passed the adversarial substantiation pipeline: it was checked to exist and to support the claim it is attached to. See how references were verified.
Registry ID ref:5900810409fa · see in the full table