Skip to content

Compact Quasi-Newton Representation

A block low-rank form of accumulated quasi-Newton Hessian or inverse-Hessian updates, built from secant-pair history to replace dense matrix operations with tall factors and a small system.

Version
v1 · 2026-09-28 · History
Domain-specific #
8582
Domain group
Formal Sciences
Origin domain
Mathematics
Subdomains
Numerical Optimization, Quasi Newton Methods → Mathematics
Aliases
Compact Representation of Quasi Newton Matrices

Core Idea

A compact quasi-Newton representation collects the effect of many recursive rank-one or rank-two updates into a single equality: an initial direct or inverse Hessian approximation plus a low-rank block correction. Its columns come from iterate displacements, gradient displacements, and update-specific combinations; a small coefficient matrix couples them.

The point is exact representation of the chosen update history, not generic compression. Matrix–vector products, solves, and some spectral calculations can work through the tall factors and small system without forming the dense approximation, which is valuable for large-scale and limited-memory optimization.

How would you explain it like I'm…

Starting Picture Plus Changes

Imagine you have a big map, and every day you make a small change to it. Instead of redrawing the whole huge map each time, you keep the first map and a short notebook of the changes. Together, they tell you exactly what the map looks like today — and you can use them without ever drawing the big map. That's a compact quasi-Newton representation.

All Updates in One Formula

Some computer methods find the lowest point of a function, like the bottom of a valley, by taking steps and updating a big table that estimates the valley's curve. After each step, the table gets a small correction. A compact quasi-Newton representation writes all those corrections at once as a starting table plus one neat block made from the steps taken and how the slope changed. It is not a rough shortcut: it gives exactly the same table. But the computer can use it without ever building the full big table, which saves lots of time and memory on huge problems.

Exact Low-Rank Hessian Update Form

Quasi-Newton optimization methods build up an approximation of the Hessian, the matrix of second derivatives describing curvature, or its inverse, by applying a small rank-one or rank-two update after each step. After many steps, this is a long chain of updates. The compact representation rewrites the whole chain as one equation: the initial approximation plus a low-rank block correction. The correction is built from tall matrices whose columns are the steps taken (changes in the position) and the changes in the gradient, plus a small coefficient matrix that ties them together. It is exact for that sequence of updates, not an approximation. Because you can multiply by it or solve with it using only these tall, thin pieces and a small system, it saves memory and time in large problems.

 

A compact quasi-Newton representation expresses the result of many recursive rank-one or rank-two quasi-Newton updates in a single closed form: an initial direct or inverse Hessian approximation plus a low-rank block correction. The correction is formed from tall matrices whose columns are iterate displacements, gradient displacements, or update-specific combinations of them, coupled through a small coefficient matrix. The equality is exact for the chosen update history; it is not a generic low-rank compression of the approximation. Matrix–vector products, linear solves (via small inner systems), and some spectral computations can be carried out through the tall factors and the small matrix without forming the dense n × n approximation. This makes the representation central to large-scale and limited-memory methods, where only a window of recent pairs is kept. The form depends on which update formula is used and on whether the direct or inverse approximation is represented.

Scope of Application

  • Numerical optimization. Applies direct or inverse quasi-Newton approximations efficiently.
  • Nonlinear equations. Represents accumulated Jacobian-like corrections.
  • Constrained methods. Supports trust-region and projected subproblems with implicit curvature.
  • Limited memory. Uses a bounded window of correction pairs.

Clarity

State whether B_k or H_k is represented, the update family, initial scaling, retained pair order, factor definitions, inner-system orientation, and curvature or nonsingularity assumptions. Inclusion test: Require an initial Hessian or inverse-Hessian model, stored secant pairs, an explicitly derived low-rank block equality, and operations through its small inner system. Exclusion test: Exclude any generic low-rank matrix, an update recursion not collected into block form, and limited-memory optimization described without the compact equality. Nearest boundary: L-BFGS also stores recent secant pairs but is an optimization algorithm and often applies an implicit inverse through two-loop recursion; the compact representation is the algebraic block representation itself. Exit condition: The object ceases to be this representation if its block factors no longer equal the stated quasi-Newton update sequence or if history is merely compressed without preserving that operator. Common misclassifications: It is not an arbitrary low-rank approximation. It is not itself a complete optimization algorithm. It is not identical to the two-loop L-BFGS recursion. Its validity depends on the selected update and admissible secant data. Nearest named distinctions: L-BFGS: An optimizer that may exploit related history but has line search, iteration, and stopping logic. Low-rank approximation: Need not reproduce recursive secant updates exactly. Sherman–Morrison–Woodbury: Is a general inverse identity rather than this history construction. Hessian compression: May be approximate rather than update-equivalent.

Manages Complexity

The representation compresses a long recursive history into two tall factors and a small algebraic core while preserving the operator implied by the update formula.

Abstract Reasoning

  1. Choose the quasi-Newton update and base approximation.
  2. Form each displacement and gradient-difference pair.
  3. Collect the history into update-specific block factors.
  4. Build and factor the small coefficient system.
  5. Apply the implied operator and monitor validity and conditioning.

Knowledge Transfer

The low-rank block identity transfers across quasi-Newton families only after the update formula, factor ordering, inner-product blocks, initial scaling, and admissibility conditions are re-derived; a visually similar factorization is not enough.

Relationships to Other Abstractions

Local relationship map for Compact Quasi-Newton RepresentationParents appear above the current abstraction, mutual partners to the right, and children below. Node labels state whether each abstraction is prime or domain-specific; colors identify relation types.Compact Quasi-NewtonRepresentationDOMAINPrime abstraction: Compression — is a kind ofCompressionPRIME

Current abstraction Compact Quasi-Newton Representation Domain-specific

Parents (1) — more general patterns this builds on

  • Compact Quasi-Newton Representation is a kind of Compression Prime

    Compact Quasi-Newton Representation is a strict kind of Compression: it stores accumulated Hessian updates in a compact block low-rank form.

Hierarchy paths (3) — routes to 3 parentless roots

Neighborhood in Abstraction Space

Compact Quasi-Newton Representation sits in a crowded region of the domain-specific corpus (40th percentile for distinctiveness): several abstractions share nearly its structure, so a description that fits it tends to fit its neighbors too.

Family — Matrices, Measures & Numeric Structures (30 abstractions)

Nearest neighbors

Computed from structural-signature embeddings · 2026-10-08