Skip to content

Full-Rank Eigendecomposition

Decomposition method — instantiates Independent Generating Set Design

Factors a square operator into its own eigenbasis, yielding a complete set of directions the operator merely rescales — and, when full-rank, a basis that spans the whole space.

Full-Rank Eigendecomposition derives a basis not from an external prescription but from a given operator itself: it finds the directions the operator only stretches or shrinks — its eigenvectors — and the factors it stretches them by — its eigenvalues. Its defining move is that this basis is the operator's own natural coordinate system, the one in which the transformation becomes diagonal (a separate independent scaling along each axis). When the operator is full-rank — no zero eigenvalue, a complete set of independent eigenvectors — those eigenvectors span the whole space, and the count of nonzero eigenvalues reads off the rank directly. Where a prescribed basis like Fourier ignores the operator, and an extraction routine trims an arbitrary set, eigendecomposition lets the operator dictate the basis that diagonalizes it.

Example

A team models how visitors move between the sections of a website as a Markov chain: a transition matrix whose entry gives the probability of hopping from one section to the next. Raw, the matrix is an opaque tangle of coupled flows. Full-Rank Eigendecomposition untangles it. The dominant eigenvector — the one with eigenvalue 1 — is the stationary distribution: where visitors pile up in the long run, the same quantity PageRank reports. The subdominant eigenvalues, all with magnitude below 1, each name an independent transient mode and how fast it decays, so the second-largest sets how quickly any starting pattern washes out toward steady state.

Because the chain is irreducible, the eigenvectors are complete and independent — a genuine basis. Any initial distribution of visitors is written as a combination of these modes, and the chain's evolution becomes trivial in that basis: each mode just decays at its own rate. The tangle became a sum of independent, individually understandable pieces.

How it works

  • Solve for the spectrum. Find the eigenvalues (the scalars for which the operator minus that scalar is singular) and their eigenvectors — algebraically for small operators, by iterative methods (QR algorithm, power iteration) at scale.
  • Assemble the eigenbasis. Stack the eigenvectors as the columns of a change-of-basis matrix; in that basis the operator is diagonal.
  • Read completeness and rank. A full set of independent eigenvectors certifies the basis spans; the number of nonzero eigenvalues is the rank.

What sets it apart is that the operator supplies the generators — the basis is discovered inside the transformation, not imposed on it.

Tuning parameters

  • Symmetric/normal vs. general operator — a symmetric operator yields orthogonal eigenvectors and a real, stable spectrum; a general one may give complex, non-orthogonal, ill-conditioned eigenvectors.
  • Full vs. partial spectrum — compute every eigenpair, or just the leading few when only the dominant modes matter.
  • Handling repeated eigenvalues — how to choose a basis within a degenerate eigenspace, where the eigenvectors are not unique.
  • Algebraic vs. numerical solution — the exact characteristic polynomial for tiny cases, iterative eigensolvers for realistic sizes.
  • Mode ordering — by eigenvalue magnitude (dominance, mixing speed) or by sign/phase, depending on what the modes are read for.

When it helps, and when it misleads

Its strength is that it gives the operator's intrinsic coordinates: a coupled linear system decouples into independent modes, powers and exponentials of the operator become trivial, and the rank and null space fall out of the spectrum. When you own the operator and it is well-behaved, no basis fits it better.

It misleads when the operator is not diagonalizable or not normal. A defective matrix has too few independent eigenvectors and simply has no eigenbasis[1]; and for a strongly non-normal operator the eigenvectors can be nearly parallel, so the eigenvalues badly mispredict transient behaviour even though each looks stable. The classic misuse is trusting eigenvalues alone for a non-normal system and missing large transient growth. The discipline is to check the conditioning of the eigenvector matrix, and to switch to the singular value decomposition — via Singular-Value Rank Diagnosis — for rectangular or rank-deficient problems where an eigenbasis does not even exist.

How it implements the components

Full-Rank Eigendecomposition fills the operator-derived, complete-basis side of the archetype:

  • candidate_generator_pool — the eigenvectors are the generators, produced by the operator rather than drawn from an external pool.
  • span_and_coverage_criterion — full rank (a complete set of independent eigenvectors) guarantees the eigenbasis covers the whole space.
  • dimension_and_rank_reference — the number of nonzero eigenvalues is the rank and fixes the dimension the operator can reach.

It does not test or prune an arbitrary candidate set for redundancy — Pivoted Row Reduction does — nor diagnose numerical rank under noise (Singular-Value Rank Diagnosis), nor orthonormalize a non-orthogonal eigenset (Gram–Schmidt Orthonormalization).

Notes

Principal component analysis is this mechanism applied to a data-derived covariance operator: the principal axes are its eigenvectors. That is the seam with Data-Adapted Basis Learning — eigendecomposition supplies the algebraic factorization, while the choice to build the operator from data and read its top modes as a learned basis is the learning mechanism's contribution, not this one's.

References

[1] A defective matrix has an eigenvalue whose geometric multiplicity is less than its algebraic multiplicity, so it lacks a full set of independent eigenvectors and cannot be diagonalized. The Jordan normal form is the generalization that covers such operators. ↩