Dying ReLU and Initialization¶
Lu, L., Shin, Y., Su, Y., & Em Karniadakis, G. (2020). Dying ReLU and Initialization: Theory and Numerical Examples. Communications in Computational Physics, 28(5), 1671-1706.
Cited by¶
1 citation across 1 artifact.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Stock Disabled Control
- The same shape governs machine-learning models collapsed into degenerate loss basins (unresponsive to learning-rate adjustment until reinitialized), degraded soil unresponsive to fertilizer until biologically rebuilt, and knowledge organizations that have lost institutional memory and do not respond to better project management.
This sourceTogether they support the ML and soil sub-claims of this composite; the institutional-memory sub-claim is not separately sourced.
Supported in partVerified against the work's full text
Lu et al. prove a dead ReLU network cannot be rescued by any gradient-based optimizer and propose re-initialization, supporting only the machine-learning sub-claim.
“Once the network is BD, we have no hope to train the network successfully.”
- The same shape governs machine-learning models collapsed into degenerate loss basins (unresponsive to learning-rate adjustment until reinitialized), degraded soil unresponsive to fertilizer until biologically rebuilt, and knowledge organizations that have lost institutional memory and do not respond to better project management.
Verification¶
Does it exist? Confirmed. This work's DOI resolves to a registered record, which fixes its identity. That is all it fixes.
Does it back the claim? Read against the text for 1 of 1 citation: 1 supported in part. Each verdict is shown under its citation below, with what in the work backs the sentence.
Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.
See how references were verified.
Links previously used in the corpus¶
Before the registry existed this work was also linked 1 other way.
Registry ID ref:cef5fc621e76 · see in the full table