Stratified Balance Check¶
Procedure — instantiates Baseline Covariate Balance Verification
Verifies covariate balance within each stratum, block, cluster, or site — at the true unit of assignment — instead of trusting a pooled comparison that can hide local imbalance.
A Stratified Balance Check re-asks the balance question inside each stratum, block, cluster, or site, and at the level where assignment actually happened, rather than only in the pooled sample. Its defining insight is that aggregate balance and local balance are different claims: two arms can look identical when all subjects are lumped together while individual clusters are badly lopsided, and — the subtler trap — a pooled table computed at the person level is simply the wrong test when randomization was done at the village or clinic level. This mechanism computes balance metrics within each stratum and at the genuine unit of assignment, turning a single reassuring average into a panel that exposes where comparability actually holds and where it quietly fails.
Example¶
A development trial testing a household water-treatment program cluster-randomizes 24 villages to treatment or control. The individual-level baseline table, built over thousands of residents, looks beautifully balanced on age, income, and prior diarrheal illness. But assignment was by village, not by person, so the effective sample is 24 units, not thousands. Re-running the check at the village level tells a different story: the twelve treatment villages happen to be systematically larger and wealthier than the twelve control villages — a real imbalance the pooled person-level table averaged into invisibility. Because the water program's benefit plausibly depends on village wealth, that stratum-level finding reshapes the whole analysis, where the individual-level table would have waved it through.
How it works¶
- Identify the true assignment unit — person, household, cluster, site, or time block — and treat it, not a convenient finer level, as the row of analysis.
- Partition by stratum (block, cluster, site, or prespecified subgroup) and recompute balance metrics within each.
- Assemble a panel across strata, so a covariate balanced on average but split lopsidedly across strata becomes visible.
- Flag strata that breach tolerance, watching specifically for the unit-of-analysis error — checking at a level finer than assignment inflates the apparent sample and manufactures false comparability.
Tuning parameters¶
- Stratifying variable(s) — which factor defines the strata (site, block, region, risk tier). More strata expose more local structure but thin each cell.
- Unit of aggregation — person-level vs. cluster-level metrics. Matching this to the assignment unit is the whole point; getting it wrong is the classic error.
- Per-stratum vs. pooled-within threshold — whether each stratum is judged individually or on a summary across strata. Individual judgment is stricter but multiplicity-prone.
- Number of strata — how finely to slice. Finer slicing catches localized imbalance but invites chance breaches somewhere.
When it helps, and when it misleads¶
Its strength is catching two failures the pooled check is blind to: localized cluster imbalance masked by a comfortable aggregate, and the wrong-level check that treats a cluster trial as if individuals were randomized. In clustered, blocked, and multi-site designs, this is the only mechanism that verifies balance where the design actually created it.
Its failure mode is multiplicity: slice the sample into enough strata and some stratum will look imbalanced purely by chance, tempting a hunt for problems that are not there.[n1] Thin strata are also noisy, so a dramatic-looking gap in a small cluster may be nothing. The classic misuse is the inverse — the unit-of-analysis error, analyzing a cluster-randomized trial at the individual level to borrow statistical power the design never provided. The guarding discipline is to prespecify the strata and respect the assignment unit, so the panel diagnoses real local imbalance instead of generating either false alarms or false confidence.
How it implements the components¶
subgroup_or_stratum_balance_panel— its core output: a per-stratum panel showing balance within each block, cluster, or subgroup.assignment_level_balance_panel— it checks balance at the genuine unit of assignment (cluster or site), not a convenient finer level.balance_metric_set— it computes standardized balance metrics within each stratum to populate the panel.
It does not audit whether the assignment map itself is intact — that is Randomization Integrity Audit — and it does not publish the human-readable covariate registry, which is Baseline Characteristics Table.
Related¶
- Instantiates: Baseline Covariate Balance Verification — it verifies balance at the right level of assignment, catching local imbalance the pooled check hides.
- Consumes: Standardized Mean Difference Table — the balance metric it recomputes within each stratum.
- Sibling mechanisms: Standardized Mean Difference Table · Baseline Characteristics Table · Covariate Balance Plot · Randomization Integrity Audit · Automated A/B Balance Dashboard · Prespecified Adjusted Estimation Plan · Balance Exception Report
Editorial Notes¶
Form Classification¶
Form family: Assessment, Review & Assurance
Rationale: Stratified Balance Check operates as a bounded evaluation of existing evidence or work that produces a finding or disposition because it verifies covariate balance within each stratum, block, cluster, or site — at the true unit of assignment — instead of trusting a pooled comparison that can hide local imbalance.
Independent corroboration: The frozen evidence defines Stratified Balance Check as 'Verifies covariate balance within each stratum, block, cluster, or site — at the true unit of assignment — instead of trusting a pooled comparison that can hide local imbalance', so its operative form is Assessment, Review & Assurance.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Statistics & Experimental Design
Origin pattern: Single lineage
Present-day reach: Specialized
Rationale: Checking covariate balance at assignment units is experimental design validation.
Related originating lineages:
- Data Science & Analytics — Data science, analytics, and operational monitoring supplies a parallel or contributing lineage for the mechanism's defining operation: verifies covariate balance within each stratum, block, cluster, or site — at the true unit of assignment — instead of trusting a pooled comparison that can hide local imbalance.
- Mathematics — Mathematical modeling, proof, and abstract-structure practice supplies a parallel or contributing lineage for the mechanism's defining operation: verifies covariate balance within each stratum, block, cluster, or site — at the true unit of assignment — instead of trusting a pooled comparison that can hide local imbalance.
- Medicine & Healthcare — Cluster and site trials require it.
Review resolution: The blind reviewers agree that statistics_experimental_design is the primary origin and differ only on alternate origin disagreement. I preserve every independently explained alternate from both records rather than imposing a numeric cap. I retain single_lineage because the combined evidence shows one traceable formative lineage. The broader reach of specialized records portability separately from historical provenance; encyclopedia_synthesis=false preserves the affirmative synthesis judgment where either reviewer identified one.
Review outcome: Reconciled after independent review; high confidence.
Notes¶
[n1] The unit-of-analysis error is analyzing a clustered or grouped design as if the smaller units within clusters were independently randomized, which overstates the true sample size and understates uncertainty. It is the mirror image of the multiplicity risk: over-slicing invents imbalance, while wrong-level pooling invents comparability. Both are avoided by fixing the analysis unit to the assignment unit in advance. ↩