Batch Normalization¶
Ioffe, S., & Szegedy, C. (2015). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. Proceedings of the 32nd International Conference on Machine Learning (ICML), 448-456.
Cited by¶
2 citations across 2 artifacts.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Gain Control
- In machine learning batch, layer, and group normalisation measure recent activation statistics and divisively rescale, return and advantage normalisation do the same in reinforcement learning, and adaptive optimisers apply per-parameter gain control to gradients by dividing by running variance.
This sourceIntroduces batch normalization, which measures recent activation statistics and divisively rescales — gain control on a network's signal path.
- In machine learning batch, layer, and group normalisation measure recent activation statistics and divisively rescale, return and advantage normalisation do the same in reinforcement learning, and adaptive optimisers apply per-parameter gain control to gradients by dividing by running variance.
- Neuromodulation
- The per-example or per-feature gain-and-bias pattern of batch and layer normalization moved from vision networks into transformer attention design.
This sourceIntroduces the learned per-feature gain-and-bias normalization pattern that moved from vision networks into transformer attention.
- The per-example or per-feature gain-and-bias pattern of batch and layer normalization moved from vision networks into transformer attention design.
Verification¶
This reference passed the adversarial substantiation pipeline: it was checked to exist and to support the claim it is attached to. See how references were verified.
Registry ID ref:50dd153d38ca · see in the full table