Skip to content

Factor Analysis

A latent-variable statistical model explains covariance among observed variables through fewer common factors, variable-specific loadings, and residual variation while making rotational and identification choices explicit.

Version
v2 · 2026-09-19 · History
Domain-specific #
1819
Domain group
Formal Sciences
Origin domain
Experimental Design & Statistics

Core Idea

Factor analysis models the covariance among several observed variables as arising partly from a smaller collection of unobserved common factors and partly from variable-specific residual variation. In its classical linear form,

\[ x=\mu+\Lambda f+\varepsilon, \]

where \(x\) is a \(p\)-vector of observed variables, \(f\) is an \(m\)-vector of common factors with \(m<p\), \(\Lambda\) is the loading matrix, and \(\varepsilon\) is residual or unique variation. If factors and residuals are uncorrelated, the model-implied covariance is

\[ \Sigma=\Lambda\Phi\Lambda^{\mathsf T}+\Psi, \]

How would you explain it like I'm…

Finding Hidden Helpers

If a kid is good at reading, spelling, and word games, maybe one hidden thing, like being good with words, helps with all three. Each game also has a little bit of its own luck mixed in. Factor Analysis is a way of guessing those hidden helpers from the scores.

Hidden Causes Behind Scores

Sometimes many measurements rise and fall together. Kids who do well on reading tests often do well on spelling and vocabulary too. Factor Analysis is a math tool that explains this using a few hidden 'factors' that we can't measure directly. Each test gets some of its score from the shared hidden factors and some from things special to that one test. The math finds how strongly each test is linked to each factor. The answer can be turned or rescaled in different ways, so people must be careful about what they say the factors mean.

Latent Common-Factor Model

Factor Analysis is a statistical model that explains how several observed variables vary together. It treats each variable as a mix of a few unobserved common factors plus its own unique variation. The 'loadings' say how much each variable depends on each factor. Because there are fewer factors than variables, the model makes a specific prediction about the pattern of covariances, and that prediction can be estimated and checked. It isn't just eyeballing which variables cluster together in a table of correlations. The factors are only defined up to rotation and scaling, so different rotated solutions fit the same data, and interpretation must respect that. It began in psychology but is now used in economics, biology, finance, and elsewhere.

 

Factor Analysis models the covariance among p observed variables as arising from m < p latent common factors plus variable-specific residuals. In the classical linear form, x = μ + Λf + ε, where Λ is the loading matrix, f the common factors, and ε the unique variation. If factors and residuals are uncorrelated, the implied covariance is Σ = ΛΦΛᵀ + Ψ, with Φ the factor covariance and Ψ the residual covariance, classically diagonal. Estimation fits Λ, Φ, and Ψ to sample covariances under identification constraints. Because rotating or rescaling the factors leaves Σ unchanged, loadings are only determined up to that equivalence, and interpretation must work within it. This makes it a covariance model, not a visual search for clusters in a correlation matrix. It began in psychometrics and is now used in economics, social science, biology, finance, and measurement research.

Scope of Application

Factor analysis applies when many measured variables are believed to share a smaller number of sources of covariation. In psychometrics, item responses may indicate abilities, traits, or attitudes. In economics, many time series may load on common macroeconomic movements. In biology, correlated assays may reflect shared regulatory programs. In survey construction, factor analysis helps examine whether items behave as indicators of intended constructs.

Use requires more than \(p>m\). Variables must carry informative covariance, the proposed measurement scale and distributional assumptions must support the chosen estimator, the sample must identify the parameters with adequate stability, and the common-factor residual structure must be plausible.

Clarity

Three quantities that are often blurred should be separated. A loading is an element of \(\Lambda\), expressing the relation between an indicator and a factor under the chosen scaling. A communality is the portion of an indicator’s variance attributed to common factors; under orthogonal standardized factors it is the sum of squared loadings for that indicator. A uniqueness is the residual variance in \(\Psi\), not necessarily pure measurement error.

Manages Complexity

Factor analysis replaces a dense set of pairwise relationships among \(p\) indicators with a smaller loading structure, factor covariance, and residual model. When the reduction is credible, the analyst can describe patterns in terms of shared latent dimensions instead of separately narrating \(p(p-1)/2\) covariances. This supports scale design, construct comparison, measurement-invariance testing, and parsimonious downstream modeling.

Abstract Reasoning

Rotational indeterminacy is central. In an orthogonal factor model with \(\Phi=I\), any orthogonal matrix \(T\) gives

\[ \Lambda\Lambda^{\mathsf T}=(\Lambda T)(\Lambda T)^{\mathsf T}. \]

Thus the covariance alone does not select one orientation of the factor axes. Oblique rotations allow correlated factors and transform \(\Lambda\) and \(\Phi\) together while preserving the implied covariance.

Knowledge Transfer

The same role inventory transfers across domains: indicators, common factors, loadings, unique variation, factor covariance, identification, fit, and interpretation. What changes is the substantive meaning assigned to factors and the measurement model appropriate to the indicators. A finance factor may represent shared return exposure; a psychometric factor may represent a construct; a gene-expression factor may summarize a coordinated program. None should be interpreted without domain evidence.

Relationships to Other Abstractions

Local relationship map for Factor AnalysisParents appear above the current abstraction, mutual partners to the right, and children below. Node labels state whether each abstraction is prime or domain-specific; colors identify relation types.Factor AnalysisDOMAINDomain-specific abstraction: Statistical Model — is a kind ofStatisticalModelDOMAINDomain-specific abstraction: Confirmatory factor analysis — is a kind ofConfirmatoryfactor analysisDOMAINDomain-specific abstraction: Factor Regression Model — is a kind ofFactorRegression ModelDOMAIN

Current abstraction Factor Analysis Domain-specific

Parents (1) — outgoing typed relations

  • Factor Analysis is a kind of Statistical Model Domain-specific

    The broader abstraction is Statistical Model: factor analysis declares a family of probability/covariance structures indexed by loadings, factor covariances, and residual parameters, then supports estimation and fit assessment.

Children (2) — abstractions linked here by typed relations

  • Confirmatory factor analysis Domain-specific is a kind of Factor Analysis

    Confirmatory factor analysis is factor analysis constrained by a prior hypothesized latent structure.

  • Factor Regression Model Domain-specific is a kind of Factor Analysis

    Factor Regression Model is a strict kind of Factor Analysis: it is a latent-factor analysis augmented with observed regression effects and residual error.

Hierarchy paths (4) — routes to 3 parentless roots

Neighborhood in Abstraction Space

Factor Analysis sits in a sparse region of the domain-specific corpus (88th percentile for distinctiveness): few abstractions share its structure, so a faithful description tends to retrieve it precisely.

Family — Unclustered & Miscellaneous (2769 abstractions)

Nearest neighbors

Computed from structural-signature embeddings · 2026-10-25