Formation Matrix¶
The inverse expected or observed likelihood-information matrix expresses local parameter dispersion for covariance bounds, standard errors, and likelihood asymptotics.
Core Idea¶
In likelihood-based statistical inference, a formation matrix is the inverse of an information matrix. The expected formation matrix is the inverse Fisher information, while the observed formation matrix is the inverse observed information, when those inverses exist. It converts local log-likelihood curvature—a precision-like quantity—into a dispersion-like matrix used for covariance lower bounds, maximum-likelihood standard errors, local confidence regions, and asymptotic expansions.
The inverse-information covariance role is anchored in the Cramér–Rao theory for regular unbiased estimation. For parameter vector \(\theta\), data \(y\), and log likelihood \(\ell(\theta;y)\), define score \(U(\theta)=\partial\ell/\partial\theta\), observed information.
Scope of Application¶
Formation matrices occur in regular parametric inference, generalized linear models, likelihood-ratio and saddlepoint asymptotics, information geometry, experimental design, and software that computes Hessian-based standard errors. The expected form is model-averaged at a parameter value. The observed form conditions on realized data through the actual log-likelihood curvature.
Barndorff-Nielsen and Cox place observed and expected information inside a systematic likelihood-asymptotic program. McCullagh's tensor treatment uses lower-index information and upper-index inverse arrays to express reparameterization-aware expansions. The formation role is especially visible when repeated index contraction propagates parameter uncertainty through derivative tensors.
Clarity¶
Expected and observed variants answer different questions. Expected information averages curvature over hypothetical samples under \(\theta\); observed information uses the curvature of the realized likelihood. Their inverses can differ substantially in finite samples even when both converge to the same first-order limit.
Index notation makes the inversion explicit. If \(j_{rs}\) are observed-information components, the observed formation components \(j^{st}\) satisfy
Manages Complexity¶
The formation matrix compresses a multivariate likelihood surface into an actionable local uncertainty geometry. Diagonal entries yield marginal variance approximations; off-diagonal entries encode local co-movement among parameter estimates; quadratic forms define ellipsoidal Wald regions; gradients sandwich the matrix in delta-method propagation.
This compression turns questions about “how sharply is the model identified?” into linear algebra. Small information eigenvalues become large formation eigenvalues, exposing weakly determined parameter combinations.
Abstract Reasoning¶
For \(n\) independent observations from \(N(\mu,\sigma^2)\) with known \(\sigma^2\), the information for \(\mu\) is \(i(\mu)=n/\sigma^2\). The formation value is
which equals the variance of the sample mean. The inverse therefore realizes both the Cramér–Rao bound and the exact covariance in this special efficient case.
Knowledge Transfer¶
The same formation roles transfer across likelihood models: normal means, logistic regression coefficients, survival-model parameters, spatial covariance parameters, and exponential-family coordinates. Model formulas differ, but information choice, inversion, parameter mapping, and uncertainty use remain literal.
Transfer from expected to observed formation requires labeling rather than substitution by habit. Software may return a negative-Hessian inverse, an outer-product-of-scores inverse, a sandwich covariance, or a penalized Hessian inverse. These can coincide asymptotically under correct specification but are not identical objects.
Relationships to Other Abstractions¶
Current abstraction Formation Matrix Domain-specific
Parents (1) — more general patterns this builds on
-
Formation Matrix is a kind of Matrix Domain-specific
Formation Matrix is a strict specialization of domain_specific:matrix: a square parameter-indexed array with inversion and bilinear-form operations, narrowed by a likelihood-information identity.
Hierarchy paths (5) — routes to 5 parentless roots
- Formation Matrix → Matrix → Tensor → Transformation → Function (Mapping)
- Formation Matrix → Matrix → Linearity
- Formation Matrix → Matrix → Representation → Abstraction
- Formation Matrix → Matrix → Tensor → Invariance
- Formation Matrix → Matrix → Tensor → Vector Space → Set and Membership
Neighborhood in Abstraction Space¶
Formation Matrix sits in a sparse region of the domain-specific corpus (83rd percentile for distinctiveness): few abstractions share its structure, so a faithful description tends to retrieve it precisely.
Family — Unclustered & Miscellaneous (1565 abstractions)
Nearest neighbors
- Variogram — 0.82
- Statistical manifold — 0.82
- Variational Bayesian Methods — 0.81
- Floor Effect — 0.80
- Method of Moments — 0.80
Computed from structural-signature embeddings · 2026-09-08