Deep Learning¶
Goodfellow, I., Bengio, Yoshua, & Courville, A. (2016). Deep Learning. MIT Press.
Cited by¶
8 citations across 8 artifacts.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Attractor Selection and Basin Control
- Dropout, batch normalization, and other techniques can be understood as basin-shifting strategies: they alter the effective landscape to steer learning away from poor local minima and toward basins containing better-generalizing solutions, as Goodfellow, Bengio, and Courville (2016) develop in their treatment of optimization in deep learning.
This sourceChapters on optimization and regularization develop dropout, batch normalization, and architectural choices as effective modifications of the loss landscape that steer training away from poor minima toward better-generalizing ones. SUPPORTS marker 054.
- Dropout, batch normalization, and other techniques can be understood as basin-shifting strategies: they alter the effective landscape to steer learning away from poor local minima and toward basins containing better-generalizing solutions, as Goodfellow, Bengio, and Courville (2016) develop in their treatment of optimization in deep learning.
- Constraint Release
- In machine learning, removing a regularizer reveals the model's intrinsic capacity to fit the data, including pathologies the regularizer was masking.
This sourceTreats regularization (L2/weight decay, dropout) as an active constraint on model capacity whose removal unmasks overfitting and the model's intrinsic fitting capacity (esp. ch. 7).
- In machine learning, removing a regularizer reveals the model's intrinsic capacity to fit the data, including pathologies the regularizer was masking.
- Continuity
- In computer science and machine learning, backpropagation requires differentiability (and hence continuity) of activation functions across the entire network; the design of activation functions like ReLU, sigmoid, and GELU is partly a problem of choosing between strict differentiability (sigmoid, GELU) and almost-everywhere differentiability (ReLU has a non-smooth point at zero but is continuous and Lipschitz, as discussed in Goodfellow, Bengio, and Courville (2016))
This sourceStandard reference for modern deep learning; develops the design and properties of activation functions, including the continuous-and-Lipschitz-but-not-differentiable-at-zero ReLU function, and the role of differentiability in backpropagation-based training.
- In computer science and machine learning, backpropagation requires differentiability (and hence continuity) of activation functions across the entire network; the design of activation functions like ReLU, sigmoid, and GELU is partly a problem of choosing between strict differentiability (sigmoid, GELU) and almost-everywhere differentiability (ReLU has a non-smooth point at zero but is continuous and Lipschitz, as discussed in Goodfellow, Bengio, and Courville (2016))
- Dimensionality Reduction
- Apparent patterns can be artifacts of the method rather than structure in the data. Not free from overfitting. Learned embeddings (autoencoders, learned manifold methods) can overfit the training data; cross-validation or held-out evaluation is important for methods with learnable parameters
This sourceCanonical deep-learning text; supports marker 065 (learned embeddings/autoencoders can overfit; held-out evaluation needed — Ch. 7 regularization, Ch. 14 autoencoders), marker 071 (manifold hypothesis underpins modern deep learning), and re-sourced marker 075 (learned embeddings with parameters can overfit and fail to generalize, unlike parameter-free PCA — regularization/generalization chapters).
- Apparent patterns can be artifacts of the method rather than structure in the data. Not free from overfitting. Learned embeddings (autoencoders, learned manifold methods) can overfit the training data; cross-validation or held-out evaluation is important for methods with learnable parameters
- Neuromodulation
- Machine learning — learning rate, temperature, dropout rate, batch-norm gain and bias, attention gates, and adapter layers all modulate: they carry no input content but set how content is processed.
This sourceStandard reference covering learning rate, dropout, normalization gain/bias, and softmax sampling temperature p_i = exp(z_i/T)/Σ exp(z_j/T) — the explicit scalar modulator that sharpens or flattens the output distribution with logits held fixed.
- Machine learning — learning rate, temperature, dropout rate, batch-norm gain and bias, attention gates, and adapter layers all modulate: they carry no input content but set how content is processed.
- Overfitting
This sourceCanonical deep-learning textbook: chapters on optimization and regularization develop dropout, batch normalization, and architectural choices as effective loss-landscape modifications steering training toward better-generalizing minima.
- Selectivity Window
- In machine-learning training, learning-rate ranges, curriculum orderings, and the regularisation sweet spot between underfitting and overfitting all instantiate the same bounded-discrimination structure.
This sourceCovers learning-rate ranges, curriculum ordering, and the regularization sweet spot between underfitting and overfitting as bounded-discrimination operating ranges in model training.
- In machine-learning training, learning-rate ranges, curriculum orderings, and the regularisation sweet spot between underfitting and overfitting all instantiate the same bounded-discrimination structure.
Domain-specific¶
Verification¶
This reference passed the adversarial substantiation pipeline: it was checked to exist and to support the claim it is attached to. See how references were verified.
Links previously used in the corpus¶
Before the registry existed this work was also linked 1 other way.
Registry ID ref:d70ecd9cf740 · see in the full table