Segment Stratification Table¶

Stratification register — instantiates Correlation Structure Characterization

Splits the data into meaningful subgroups and estimates the association within each, so a pattern that holds overall but reverses inside every subgroup — or vice versa — cannot hide.

A Segment Stratification Table partitions observations into discrete, decision-relevant groups — by product, region, cohort, channel, severity — and reports the association separately within each stratum alongside the pooled figure. Its defining purpose is to expose subgroup heterogeneity and reversal: the case where an aggregate association is an artifact of group mix and disappears, or flips sign, once you look inside each group. Where a partial-correlation probe conditions on continuous controls, this lays the dependence out across explicit, named segments as a governance artifact you can read row by row.

Example¶

An e-commerce team sees that shoppers who saw a promotional banner converted less than those who did not — a raw negative correlation that seems to indict the banner. The stratification table splits by acquisition channel. Within paid search, banner-viewers convert more; within organic, more; within email, more. The aggregate reversed only because the banner was shown disproportionately to low-intent organic traffic. Stratified, the banner looks good in every channel — the pooled number was a mix artifact, and the table is what made that legible before the campaign was cut.

How it works¶

Choose the segmentation. Pick the variable(s) along which dependence might genuinely differ.
Estimate within and across. Compute the association inside each stratum and in the pool, and place them side by side.
Flag departures. Mark strata whose sign or magnitude diverges from the pooled figure — a full reversal above all — and record each segment's sample size so thin, unreliable strata are visible rather than trusted.

Tuning parameters¶

Segmentation variable and granularity — coarse strata can hide heterogeneity; fine strata shrink each group until its estimate is noise. The central dial.
Minimum stratum size — the floor below which a segment's correlation is reported as unreliable rather than believed.
Pooled-versus-stratified emphasis — whether the headline is the aggregate or the within-segment figures; the paradox turns on which you privilege.
Recombination weighting — equal versus size-weighted when rolling strata back up, which changes what "overall" even means.

When it helps, and when it misleads¶

Its strength is catching Simpson's-paradox reversals and subgroup effects that a single coefficient averages away, and making the group mix — and its influence on the headline number — explicit and governable.^[1]

Its hazards are the mirror image of that power. Slice finely enough and some segment will show a striking correlation by chance (subgroup fishing, the multiple-comparisons trap), while thin strata throw off wild estimates that masquerade as signal. Choosing the segmentation after seeing the data lets you engineer whichever story you want. The classic misuse is reporting the one flattering subgroup, or slicing until an unwanted aggregate effect fragments into noise. The discipline: pre-specify segments, enforce a minimum stratum size, and treat any post-hoc subgroup as a hypothesis that owes its own test.

How it implements the components¶

conditional_and_segment_structure — fills the segment half: the association within explicit, discrete strata, where the partial-correlation probe fills the continuous-conditioning half.
sampling_frame — each stratum is a defined sub-population with its own sample; the table records which units fall in each and how many, so coverage and thin cells are auditable.

It does not perform continuous conditioning or residualizing (Partial-Correlation or Residual Probe), decompose common drivers (Covariance or Factor Model), or correct for the many subgroup tests it invites (Permutation Null and Multiplicity Check).

Instantiates: Correlation Structure Characterization — it supplies the subgroup layer that keeps an aggregate association from masking a within-group reversal.
Sibling mechanisms: Partial-Correlation or Residual Probe · Correlation Heatmap · Covariance or Factor Model · Lag-Correlation Matrix · Rolling Correlation Dashboard · Bootstrap Association Interval · Dependence-Measure Selection Matrix · Joint-Distribution Diagnostic Panel · Nonlinear Dependence Screen · Causal-Claim Labeling Template · Outlier, Range, and Transformation Sensitivity Review · Permutation Null and Multiplicity Check

Notes¶

Stratifying invites multiple comparisons: every extra segment is another test, and enough of them guarantees a false positive. The table should travel with a multiplicity control (Permutation Null and Multiplicity Check) and a pre-registered segmentation, or its most eye-catching row is as likely to be noise as news.

References¶

[1] Simpson's paradox — an association present in aggregated data that disappears or reverses when the data are split into subgroups, typically because a confounding variable drives group membership; the canonical reason to stratify before trusting a pooled correlation. ↩