The Loss Surfaces of Multilayer Networks¶
Choromanska, A., Henaff, M., Mathieu, M., Arous, G. B., & LeCun, Y. (2015). The Loss Surfaces of Multilayer Networks. Proceedings of the 18th International Conference on Artificial Intelligence and Statistics (AISTATS), 192-204.
Cited by¶
1 citation across 1 artifact.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Multi Path Convergence
- In optimization and machine learning, gradient descent from many random initializations on a non-convex landscape converges to the same basin, and the empirical fact that different seeds yield qualitatively similar trained models is multi-path convergence.
This sourceAnalyzes the loss landscape of deep networks, showing that gradient descent from many random initializations converges to qualitatively similar low-loss minima.
Supported in partVerified against the publisher's abstract
Choromanska et al. show that in large decoupled networks most low critical values of the non-convex loss lie in a narrow band just above the global minimum, which supports similar-quality minima rather than convergence to one basin.
“We show that for large-size decoupled networks the lowest critical values of the random loss function form a layered structure and they are located in a well-defined band lower-bounded by the global minimum. The number of local minima outside that band diminishes exponentially with the size of the network.”
- In optimization and machine learning, gradient descent from many random initializations on a non-convex landscape converges to the same basin, and the empirical fact that different seeds yield qualitatively similar trained models is multi-path convergence.
Verification¶
Does it exist? Not checked yet. This entry carries no identifier to resolve. It was extracted from the citation as written in the article, normalized, and deduplicated against the rest of the registry.
Does it back the claim? Read against the text for 1 of 1 citation: 1 supported in part. Each verdict is shown under its citation below, with what in the work backs the sentence.
Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.
See how references were verified.
Registry ID ref:8727106acd81 · see in the full table