Log-Sum Inequality¶
For nonnegative sequences aᵢ and bᵢ with totals A and B, the inequality ∑aᵢ log(aᵢ/bᵢ) ≥ A log(A/B), under standard zero conventions, with equality when the positive ratios aᵢ/bᵢ are constant.
Core Idea¶
The log-sum inequality says convex information cost cannot be reduced below the cost computed after aggregating corresponding masses. Local ratio variation adds nonnegative excess beyond the total ratio.
Its proof and equality condition reveal the mechanism: Jensen compares the weighted average of x log x with the function at the weighted average, and equality means all supported local ratios coincide.
Scope of Application¶
- Information theory. Proves divergence and data-processing bounds.
- Statistics. Analyzes relative entropy and likelihood ratios.
- Convex analysis. Provides an aggregation form of Jensen.
- Probability. Derives Gibbs' inequality and related inequalities.
Clarity¶
State index set, nonnegativity, totals and whether positive, log base, zero and infinity conventions, support condition, finite versus extended-real result, equality condition including zero terms, normalization, partition or aggregation, and the precise consequence being derived. Inclusion test: Require nonnegative aligned sequences, declared logarithm and zero conventions, finite or extended-real interpretation, and the log-sum comparison between coordinate and aggregated ratios. Exclusion test: Exclude log of a sum inequalities with different form, Gibbs' inequality stated without the underlying two-sequence aggregation, Jensen applied to unrelated weights, negative inputs, and algebra that silently divides by zero. Nearest boundary: Gibbs' inequality is a probability-specialized consequence obtained by totals A=B=1; log-sum is the more general nonnormalized statement. Exit condition: The statement changes with negative entries, zero totals, unsupported positive mass, logarithm base, infinite index sets, or alternative generalized functions. Common misclassifications: It is not the elementary identity log(xy)=log x+log y. Negative inputs are outside the standard statement. Zeros cannot be handled by ordinary division. Equality is not merely A=B unless local ratios also match. Nearest named distinctions: Gibbs' inequality: Is the normalized probability consequence. Jensen's inequality: Is the general convex theorem used in the proof. Sum–log inequality: May refer informally to unrelated concavity bounds for log. KL divergence: Is a quantity whose nonnegativity follows from log-sum.
Manages Complexity¶
The formula compresses support, normalization, convexity, and equality subtleties. Many incorrect applications are numerically plausible because zero terms or unnormalized measures are silently treated as probabilities.
Abstract Reasoning¶
- Verify aligned nonnegative sequences and support conditions.
- Compute totals and separate zero-total edge cases.
- Rewrite the sum using b-weights and f(x)=x log x.
- Apply Jensen under normalized weights and restore the scale B.
- Analyze equality and limiting zero cases before specializing to information quantities.
Knowledge Transfer¶
The convex aggregation pattern transfers to f-divergences and integral forms when the generating function and measure conditions are rebuilt. The logarithmic formula and equality condition should not be copied to nonconvex functions.
Relationships to Other Abstractions¶
Current abstraction Log-Sum Inequality Domain-specific
Parents (1) — more general patterns this builds on
-
Log-Sum Inequality presupposes Convexity Prime
Log-Sum Inequality presupposes Convexity: the parent's defining role is necessary to the child's frozen mechanism or criterion.
Hierarchy path (1) — routes to 1 parentless root
- Log-Sum Inequality → Convexity → Optimization
Neighborhood in Abstraction Space¶
Log-Sum Inequality sits in a crowded region of the domain-specific corpus (27th percentile for distinctiveness): several abstractions share nearly its structure, so a description that fits it tends to fit its neighbors too.
Family — Matrices, Measures & Numeric Structures (30 abstractions)
Nearest neighbors
- Canberra Distance — 0.91
- Partition problem — 0.90
- Grey Relational Analysis — 0.89
- Matrix Multiplication — 0.89
- Continued Fraction — 0.88
Computed from structural-signature embeddings · 2026-10-08