Fowlkes–Mallows Index

Compare two partitions by the geometric mean of pairwise co-membership precision and recall, rewarding pairs clustered together by both while excluding true-negative pairs from the score.

Version

v2 · 2026-09-06 · History

Domain-specific #

1874

Origin domain

statistics

Subdomain

cluster analysis

Aliases

Fowlkes–Mallows measure, FM index, Fowlkes–Mallows score, B K statistic

Core Idea

The Fowlkes–Mallows Index compares two partitions of the same finite item set through unordered item pairs. Let \(TP\) count pairs placed in the same block by both partitions, \(FP\) pairs co-clustered only by the first, and \(FN\) pairs co-clustered only by the second. When both denominators are positive, \(\mathrm{FM}=TP/\sqrt{(TP+FP)(TP+FN)}\). Equivalently, it is \(\sqrt{PR}\), the geometric mean of pairwise precision \(P=TP/(TP+FP)\) and pairwise recall \(R=TP/(TP+FN)\). The score lies in \([0,1]\); one means the two partitions induce identical pairwise co-membership relations.^[1]

Fowlkes and Mallows introduced the measure for comparing two hierarchical clusterings after cutting both trees into \(k\) clusters. If \(m_{ij}\) is the contingency table between the two cuts and \(m_{i+},m_{+j}\) are its margins, their quantities can be written \(T_k=\sum_{ij}m_{ij}^{2}-n\), \(P_k=\sum_i m_{i+}^{2}-n\), and \(Q_k=\sum_j m_{+j}^{2}-n\), with \(B_k=T_k/\sqrt{P_kQ_k}\). These counts are twice the corresponding unordered-pair counts, so the common factor cancels. A plot of \((k,B_k)\) displays agreement across hierarchy levels rather than collapsing the comparison to one arbitrary cut.^[1]

The index ignores true negatives: the often enormous set of pairs separated by both partitions does not directly raise the score. This makes it less dominated by joint separation than the unadjusted Rand index when most item pairs lie in different clusters. The choice is not universally superior. FM is sensitive to cluster-size distribution, has no intrinsic correction for agreement expected under a random-partition model, and can change when many singleton or highly imbalanced blocks are introduced. Hubert and Arabie's analysis of partition comparison shows why chance correction and the assumed random model are separate methodological decisions.^[2]

Despite occasional loose usage, FM is a similarity index, not generally a mathematical metric: it reports greater value for greater agreement, has zero rather than one at maximal dissimilarity, and no distance derived as \(1-\mathrm{FM}\) is guaranteed to satisfy the triangle inequality. It is also symmetric in the two partitions even though its precision and recall factors exchange roles. Degenerate partitions can make a denominator zero—for example, if one partition has no co-clustered pairs. Software must declare whether the result is undefined, assigned by convention, or handled through a special case. The stable abstraction is the pairwise geometric-mean comparison with an explicit counting and degeneracy convention.

Structural Signature

• Common item universe. Both partitions classify exactly the same declared set of observations.
• Two partitions. Each item belongs to one block in each hard, nonoverlapping partition.
• Pair universe. Unordered distinct item pairs are the elementary comparison units.
• Joint co-membership count. True positives are pairs joined by both partitions.
• First-only co-membership count. False positives are joined only by the first partition.
• Second-only co-membership count. False negatives are joined only by the second partition.
• True-negative exclusion. Pairs separated in both partitions do not enter the formula.
• Precision factor. Jointly joined pairs are normalized by all pairs joined in the first partition.
• Recall factor. Jointly joined pairs are normalized by all pairs joined in the second partition.
• Geometric aggregation. The index is the square root of the factor product.
• Hierarchy convention. For dendrograms, the cut level and whether equal \(k\) is required are stated.
• Degeneracy policy. Zero denominators, empty universes, missing items, and weighting are handled explicitly.

What It Is Not

• Not a distance metric. The similarity itself does not satisfy distance identity or triangle-inequality requirements.
• Not the Rand Index. Rand additionally counts pairs separated by both partitions.
• Not the Adjusted Rand Index. FM has no built-in chance correction.
• Not silhouette width. Silhouette is an internal index using dissimilarities, not an external comparison of partitions.
• Not a clustering algorithm. It evaluates or compares outputs; it does not construct clusters.
• Not label accuracy. Cluster labels can be permuted without changing the score.
• Not limited to a reference truth. The two arguments are symmetric and may both be estimated clusterings.
• Not automatically defined for every partition pair. All-singleton and one-block edge cases require a declared convention.

Scope of Application

The Fowlkes–Mallows Index is literal when two hard partitions of the same finite objects are compared by agreement on which pairs belong together.

• External cluster validation. An estimated partition is compared with a benchmark classification.
• Method comparison. Outputs from different clustering algorithms are compared without matching arbitrary labels.
• Stability analysis. Bootstrap or perturbation runs are compared pairwise to assess co-membership stability.
• Hierarchical clustering. Dendrogram cuts are compared over a range of cluster counts using \(B_k\) curves.
• Consensus clustering. Candidate consensus partitions are scored against contributing partitions.
• Image segmentation. Region partitions of a common pixel or object universe are externally compared.
• Bioinformatics. Sample, cell, gene, or sequence partitions are compared under a fixed item universe.
• Benchmark reporting. FM complements indices with different true-negative and chance-baseline behavior.

Clarity

Declare the common item universe, whether pairs are unordered and exclude self-pairs, which partition supplies the precision denominator, the hard-partition assumption, and the zero-denominator convention. Report \(TP\), \(FP\), and \(FN\) or the contingency table alongside the score so it can be reproduced. For hierarchies, name every cut rule and cluster count. Do not call FM a metric without defining and proving a separate distance. Do not interpret a raw value against a universal chance baseline; the expected value depends on the cluster-size and random-partition model. If observations are weighted, missing, overlapping, fuzzy, or multi-label, specify an extension rather than silently applying the hard-partition formula.

Manages Complexity

Comparing partitions naively requires matching arbitrary block labels and reasoning over whole set systems. FM removes label identity by reducing both partitions to the same binary relation on item pairs, then compresses three relevant counts into a symmetric scalar. The geometric mean balances one-sided over-merging and over-splitting: high precision with low recall, or the reverse, cannot yield a high final score. This compression hides structure. Different error patterns can share one value, large blocks contribute quadratically many pairs, and the omission of true negatives changes behavior relative to Rand-type indices. Responsible use therefore preserves contingency counts, cluster-size profiles, hierarchy levels, uncertainty, and companion metrics rather than treating FM as a complete evaluation.

Abstract Reasoning

1. Verify that both inputs partition the same item universe.
2. Enumerate or derive counts for every unordered pair's two co-membership decisions.
3. Compute \(TP\), \(FP\), and \(FN\) without using arbitrary cluster labels.
4. Check whether \(TP+FP\) and \(TP+FN\) are positive.
5. Apply and disclose the chosen convention if either denominator vanishes.
6. Compute pairwise precision and recall, then their geometric mean.
7. Verify symmetry by swapping partitions and observing that the factors exchange.
8. Inspect cluster-size imbalance and the contributions of large blocks.
9. Compare with a chance-adjusted or true-negative-inclusive index when the decision requires it.
10. For hierarchies, repeat at declared cut levels and interpret the full curve.

Knowledge Transfer

The strict parent is Comparison. FM takes two structured outputs on a common universe and produces a rule-governed similarity judgment from their relational agreements and disagreements. The parent transfers the need for aligned objects, declared criteria, and interpretable direction. The domain residual is pairwise co-membership, exclusion of true negatives, and geometric aggregation of precision and recall. Clustering is the object domain but cannot be a specialization parent because FM compares clusterings rather than constructing one.

Examples

Canonical

For four items, let partition \(A=\{\{1,2,3\},\{4\}\}\) and partition \(B=\{\{1,2\},\{3,4\}\}\). Partition \(A\) joins pairs \(12,13,23\); \(B\) joins \(12,34\). Thus \(TP=1\), \(FP=2\), and \(FN=1\). The index is \(1/\sqrt{3\cdot2}=1/\sqrt 6\approx0.408\). The four jointly separated pairs do not enter the score. Relabeling either partition's blocks leaves every count unchanged.

Mapped back: two hard partitions → pairwise co-membership table → precision and recall → geometric-mean similarity.

Applied / In Practice

A laboratory clusters the same 2,000 cells after two preprocessing pipelines. One produces a large heterogeneous cluster while the other splits it. The analysts report FM with the pair counts, Adjusted Rand Index, cluster-size distribution, and a cross-tabulation. FM exposes asymmetric pair joining without rewarding the many pairs separated by both runs, while the companion results show how much agreement could arise from the chosen partition structure.^[2]

Mapped back: matched items + competing clusterings → transparent pair counts → FM plus baseline-sensitive companions → qualified stability conclusion.

Structural Tensions

• Co-membership focus vs. joint-separation evidence. Excluding true negatives prevents domination but discards agreement. Diagnostic: Is separating most pairs substantively informative in this task?
• Symmetry vs. benchmark direction. The score is symmetric even when one partition is called truth. Diagnostic: Are one-sided costs needed in addition to FM?
• Raw agreement vs. chance baseline. Cluster sizes alter expected similarity. Diagnostic: Which random-partition model matches the data-generating procedure?
• Scalar simplicity vs. error localization. One number hides which blocks merge or split. Diagnostic: Have the contingency table and per-block errors been inspected?
• Pair weighting vs. cluster weighting. Large clusters contribute many more pairs. Diagnostic: Should each pair, item, or cluster carry equal decision weight?
• Autonomous residual vs. generic Comparison. Many scores compare two outputs. Diagnostic: Are pairwise co-membership precision and recall geometrically combined with true negatives excluded?

Structural–Framed Character

A common item universe, two hard partitions, unordered pair decisions, \(TP/FP/FN\) counts, true-negative exclusion, and geometric aggregation are structural. Item type, cluster algorithm, benchmark status, hierarchy cut, software, weighting, and reporting context are framed. FM guarantees a reproducible similarity under its conventions; it does not guarantee statistical significance, chance adjustment, metric geometry, clustering quality, or substantive validity of either partition.

Structural Core vs. Domain Accent

The transferable skeleton is Comparison: align two objects and aggregate matched and mismatched relations. The cluster-analysis accent is partition-induced pair co-membership, precision/recall normalization, geometric mean, hierarchy cuts, and label invariance. Removing that accent yields a generic similarity measure; adding true negatives yields Rand-type indices; chance correction requires another construction.

Instantiates / Related Primes

Comparison is the strict parent by composition/presupposition: FM compares two partitions under a fixed relational criterion and returns an ordered similarity judgment. The edge is not to Metric because FM is a similarity without the metric axioms, and not to Clustering because the index evaluates clusterings rather than producing them.

The prospective workspace queue contains one strict upward edge to prime:comparison. No live DAG mutation is authorized.

Relationships to Other Abstractions

Current abstraction Fowlkes–Mallows Index Domain-specific

Parents (1) — more general patterns this builds on

• Fowlkes–Mallows Index is a kind of Comparison Prime

  Comparison is the strict parent by composition/presupposition: FM compares two partitions under a fixed relational criterion and returns an ordered similarity judgment.

Hierarchy path (1) — routes to 1 parentless root

• Fowlkes–Mallows Index → Comparison → Self Checking

Neighborhood in Abstraction Space

Fowlkes–Mallows Index sits in a sparse region of the domain-specific corpus (97^th percentile for distinctiveness): few abstractions share its structure, so a faithful description tends to retrieve it precisely.

Family — Unclustered & Miscellaneous (1565 abstractions)

Nearest neighbors

• Disjoint-Set Data Structure — 0.77
• Wigner Surmise — 0.77
• Wilf Equivalence — 0.75
• Unified field theory — 0.75
• Gower's Distance — 0.75

Computed from structural-signature embeddings · 2026-09-08

Not to Be Confused With

• Rand Index. Counts agreement on both joined and separated pairs.
• Adjusted Rand Index. Corrects a Rand-type score under a specified random model.
• Jaccard Index for Partitions. Uses \(TP/(TP+FP+FN)\) rather than a geometric mean.
• F1 Score. Uses the harmonic mean of precision and recall rather than the geometric mean.
• Mutual Information. Compares partition label variables through information dependence.
• Silhouette Coefficient. Uses within- and between-cluster dissimilarities without a second partition.
• Matthews Correlation Coefficient. A confusion-matrix correlation including true negatives.

References

[1] E. B. Fowlkes and C. L. Mallows, “A Method for Comparing Two Hierarchical Clusterings,” Journal of the American Statistical Association 78, no. 383 (1983): 553–569, https://doi.org/10.1080/01621459.1983.10478008. registry ↩a ↩b

[2] Lawrence Hubert and Phipps Arabie, “Comparing Partitions,” Journal of Classification 2 (1985): 193–218, https://doi.org/10.1007/BF01908075. registry ↩a ↩b