Large width limits of neural networks¶

Asymptotic regimes in which neural-network layer widths tend to infinity and random networks converge to analytically tractable Gaussian-process, kernel, mean-field, or feature-learning descriptions.

Version: v1 · 2026-09-08 · History
Domain-specific #: 5263
Origin domain: theoretical machine learning
Subdomain: theoretical machine learning

Core Idea¶

Large-width limits replace a finite interacting parameter system with a limiting stochastic process or deterministic evolution, but the result depends on architecture, initialization and learning-rate scaling, training rule, depth, and order of limits. Width-dependent random sums concentrate or satisfy central-limit behavior under a declared parameterization; training dynamics then converge to a kernel, feature distribution, or mean-field equation on a specified time scale. The abstraction is therefore identified by a declared carrier, a transformation or constraint over that carrier, and an invariant that tells an analyst whether the named structure is genuinely present.

Scope of Application¶

Large width limits of neural networks belongs to theoretical machine learning and is useful where the analyst can specify the typed theoretical machine learning carrier, defining objects and relations, parameters, conventions, evidence, boundary cases, and comparison targets, then evaluate the architecture sequence, width parameter, initialization law and variance scaling, depth regime, optimizer and learning-rate scaling, convergence mode, order of limits, and finite-width error are explicit. The scope is broad within that domain but bounded by the need for the architecture sequence, width parameter, initialization law and variance scaling, depth regime, optimizer and learning-rate scaling, convergence mode, order of limits, and finite-width error are explicit.

Clarity¶

The abstraction clarifies a crowded vocabulary by making the architecture sequence, width parameter, initialization law and variance scaling, depth regime, optimizer and learning-rate scaling, convergence mode, order of limits, and finite-width error are explicit the center of the account. A claim should name the carrier, the governing operation or relation, the applicable assumptions, and the recognition test.

Manages Complexity¶

Without the abstraction, an analyst must reason directly over many local details: the carrier roles, admissibility assumptions, competing conventions, derived invariants, boundary cases, and proof or validation obligations specific to Large width limits of neural networks. Large width limits of neural networks compresses them into the roles in the structural signature. That compression permits comparison across instances without erasing the variables that determine validity. It also exposes which details may be varied safely and which are constitutive.

Abstract Reasoning¶

Identify the carrier. State what the elements, states, objects, or observations are: the typed theoretical machine learning carrier, defining objects and relations, parameters, conventions, evidence, boundary cases, and comparison targets. Reject examples whose alleged carrier belongs to a different problem. 2. Lock the constitutive rule. Express the architecture sequence, width parameter, initialization law and variance scaling, depth regime, optimizer and learning-rate scaling, convergence mode, order of limits, and finite-width error are explicit independently of one notation or implementation.

Knowledge Transfer¶

Knowledge transfers strongly among subfields of theoretical machine learning because they reuse the typed theoretical machine learning carrier, defining objects and relations, parameters, conventions, evidence, boundary cases, and comparison targets, Width-dependent random sums concentrate or satisfy central-limit behavior under a declared parameterization; training dynamics then converge to a kernel, feature distribution, or mean-field equation on a specified time scale., and type the carrier, state every parameter and convention in the definition, test that the architecture sequence, width parameter, initialization law and variance scaling, depth regime, optimizer and learning-rate scaling, convergence mode, order of limits, and finite-width error are explicit, compare the nearest accepted identity, and report counterexamples, uncertainty, and limiting cases.

Relationships to Other Abstractions¶

Current abstraction Large width limits of neural networks Domain-specific

Parents (1) — more general patterns this builds on

Large width limits of neural networks is a kind of Scaling and Scale Dependence Prime

The proposed strict upward parent is prime:scaling_and_scale_dependence.

Hierarchy path (1) — routes to 1 parentless root

Large width limits of neural networks → Scaling and Scale Dependence → Scale

Neighborhood in Abstraction Space¶

Large width limits of neural networks sits in a crowded region of the domain-specific corpus (22^nd percentile for distinctiveness): several abstractions share nearly its structure, so a description that fits it tends to fit its neighbors too.

Family — Machine Learning & Statistical Estimation (24 abstractions)

Nearest neighbors

Computed from structural-signature embeddings · 2026-09-08