Factor Analysis

A latent-variable statistical model explains covariance among observed variables through fewer common factors, variable-specific loadings, and residual variation while making rotational and identification choices explicit.

Version

v1 · 2026-08-30 · History

Domain-specific #

1819

Origin domain

statistics

Core Idea

Factor analysis models the covariance among several observed variables as arising partly from a smaller collection of unobserved common factors and partly from variable-specific residual variation. In its classical linear form,

\[ x=\mu+\Lambda f+\varepsilon, \]

where \(x\) is a \(p\)-vector of observed variables, \(f\) is an \(m\)-vector of common factors with \(m<p\), \(\Lambda\) is the loading matrix, and \(\varepsilon\) is residual or unique variation. If factors and residuals are uncorrelated, the model-implied covariance is

\[ \Sigma=\Lambda\Phi\Lambda^{\mathsf T}+\Psi, \]

Scope of Application

Factor analysis applies when many measured variables are believed to share a smaller number of sources of covariation. In psychometrics, item responses may indicate abilities, traits, or attitudes. In economics, many time series may load on common macroeconomic movements. In biology, correlated assays may reflect shared regulatory programs. In survey construction, factor analysis helps examine whether items behave as indicators of intended constructs.

Use requires more than \(p>m\). Variables must carry informative covariance, the proposed measurement scale and distributional assumptions must support the chosen estimator, the sample must identify the parameters with adequate stability, and the common-factor residual structure must be plausible.

Clarity

Three quantities that are often blurred should be separated. A loading is an element of \(\Lambda\), expressing the relation between an indicator and a factor under the chosen scaling. A communality is the portion of an indicator’s variance attributed to common factors; under orthogonal standardized factors it is the sum of squared loadings for that indicator. A uniqueness is the residual variance in \(\Psi\), not necessarily pure measurement error.

Manages Complexity

Factor analysis replaces a dense set of pairwise relationships among \(p\) indicators with a smaller loading structure, factor covariance, and residual model. When the reduction is credible, the analyst can describe patterns in terms of shared latent dimensions instead of separately narrating \(p(p-1)/2\) covariances. This supports scale design, construct comparison, measurement-invariance testing, and parsimonious downstream modeling.

Abstract Reasoning

Rotational indeterminacy is central. In an orthogonal factor model with \(\Phi=I\), any orthogonal matrix \(T\) gives

\[ \Lambda\Lambda^{\mathsf T}=(\Lambda T)(\Lambda T)^{\mathsf T}. \]

Thus the covariance alone does not select one orientation of the factor axes. Oblique rotations allow correlated factors and transform \(\Lambda\) and \(\Phi\) together while preserving the implied covariance.

Knowledge Transfer

The same role inventory transfers across domains: indicators, common factors, loadings, unique variation, factor covariance, identification, fit, and interpretation. What changes is the substantive meaning assigned to factors and the measurement model appropriate to the indicators. A finance factor may represent shared return exposure; a psychometric factor may represent a construct; a gene-expression factor may summarize a coordinated program. None should be interpreted without domain evidence.

Relationships to Other Abstractions

Current abstraction Factor Analysis Domain-specific

Parents (1) — more general patterns this builds on

• Factor Analysis is a kind of Statistical Model Domain-specific

  The proposed parent is Statistical Model: factor analysis declares a family of probability/covariance structures indexed by loadings, factor covariances, and residual parameters, then supports estimation and fit assessment.

Hierarchy paths (6) — routes to 4 parentless roots

• Factor Analysis → Statistical Model → Representation → Abstraction

Show alternative paths (5)

Neighborhood in Abstraction Space

Factor Analysis sits in a sparse region of the domain-specific corpus (89^th percentile for distinctiveness): few abstractions share its structure, so a faithful description tends to retrieve it precisely.

Family — Unclustered & Miscellaneous (1565 abstractions)

Nearest neighbors

• Measurement Invariance — 0.82
• Latent growth modeling — 0.79
• Collinearity Inflation — 0.79
• Sargan–Hansen Test — 0.79
• Suppressor variable — 0.78

Computed from structural-signature embeddings · 2026-09-08