Stratified Analysis Protocol¶
Analysis protocol — instantiates Aggregation Bias Detection and Correction
Splits an aggregate into pre-declared strata and compares each subgroup's pattern against the pooled figure, so hidden heterogeneity surfaces before the claim is trusted.
Before an aggregate is allowed to carry a decision, Stratified Analysis Protocol breaks it into a small, named-in-advance set of strata and compares each stratum back to the pool. Its defining move is the pre-declaration: the analyst commits — before looking — to which partitions could change interpretation and to a rule for reading each slice against the whole. That single discipline is what separates honest stratification from opportunistic slicing. The protocol does not decide which level is causally right or reweight anything; it produces the audited breakdown that the rest of the correction loop reads. It is the substrate: a disciplined table of stratum-specific estimates, each tagged with how many observations stand behind it.
Example¶
A trial is testing an antiviral against a respiratory infection, and the headline is an overall efficacy of roughly 55%. Before publishing "the drug works," the team runs the stratified protocol they registered at the outset. Three strata were pre-declared as capable of changing the reading: baseline severity (mild / moderate / severe), age band, and hours from symptom onset to first dose. Disaggregated, the picture fractures: mild-to-moderate patients dosed early show around 70% efficacy, while the severe stratum sits near 15%. The severe cell, though, holds only about 40 patients — below the pre-set floor — so its estimate is flagged as unstable rather than reported as a finding. The outcome is not "the drug works" but a bounded claim: start early, in mild-to-moderate disease, with the severe question marked "underpowered, collect more." The aggregate survived, but only after being narrowed to the population it actually describes.
How it works¶
- Pre-register the strata. Name the partitions and the order they'll be examined before seeing results, so a dramatic slice can't be reverse-selected later.
- Compute stratum-specific estimates. Produce the metric within each cell alongside the pooled value.
- Apply the comparison rule. For each stratum, classify it against the aggregate: agrees, attenuates, exaggerates, or reverses. Divergence is the signal to hand downstream.
- Enforce the cell-size floor. Any stratum below the minimum count is collapsed with a neighbor or suppressed and marked, never reported as a clean estimate.
- Optionally re-pool. Where a single summary is still wanted, combine strata with a method that holds composition fixed (e.g. Cochran–Mantel–Haenszel[n1]) rather than re-averaging raw counts.
Tuning parameters¶
- Stratum set — how many and which partitions to declare. More strata catch finer heterogeneity but multiply comparisons and thin every cell.
- Cell-size floor — the minimum count a stratum needs to report. A higher floor kills noise but hides small, possibly high-stakes groups.
- Comparison metric — absolute vs. relative difference from the pool. Relative differences dramatize small-base strata; absolute ones can bury them.
- Pre-registration strictness — hard lock vs. amendable plan. Strict locks bar fishing but can't adapt to a surprise the data reveal.
- Re-pooling method — raw vs. composition-adjusted summary. Adjustment corrects composition but is harder to explain.
When it helps, and when it misleads¶
It is at its best when the subgroup dimensions that matter are known in advance and each cell has enough data to estimate — exactly the conditions under which a pooled number can quietly average away a group in trouble. Its honest failure mode is what happens when the pre-declaration slips: without it, stratification degenerates into multiple-comparison fishing, where enough slices guarantee a "significant" one by chance. The classic misuse is the trial subgroup mined after the fact — "the drug helped left-handed patients over 60" — a spurious slice paraded as a finding. The guarding discipline is to fix the strata and the cell-size floor in advance, adjust for multiplicity, and treat any unregistered slice as hypothesis-generating only.
How it implements the components¶
subgroup_structure_map— the pre-declared list of strata is the map of partitions that could change interpretation.disaggregation_plan— the registered protocol (which strata, in what order, with what thresholds) is the plan that keeps review from being absent or opportunistic.aggregate_subgroup_comparison_rule— the agrees / attenuates / exaggerates / reverses classification per stratum turns the breakdown into diagnosis.minimum_cell_size_guardrail— the enforced floor that suppresses under-powered cells.
It does not weight strata to a target population — that belongs to Poststratification or Reweighting — and it does not record a family of alternative specifications, which is Sensitivity Analysis by Group's job.
Related¶
- Instantiates: Aggregation Bias Detection and Correction — supplies the audited stratified breakdown the correction loop reads.
- Compare: Stratified Analysis shares the split-into-strata move but is a confounder-adjustment estimation method — output = the de-confounded effect, catching a Simpson reversal. This protocol is instead a pre-registered subgroup-audit discipline: output = an audited breakdown table with a cell-size floor, diagnosis-agnostic.
- Sibling mechanisms: Simpson's Paradox Check · Ecological Fallacy Guardrail · Poststratification or Reweighting · Multilevel Modeling Review · Sensitivity Analysis by Group · Subgroup Dashboard with Warning Flags · Representativeness and Nonresponse Review
Editorial Notes¶
Form Classification¶
Form family: Analysis, Modeling & Optimization
Rationale: Stratified Analysis Protocol operates as an analytical, modeling, inference, comparison, or optimization procedure that derives insight or a solution because it splits an aggregate into pre-declared strata and compares each subgroup's pattern against the pooled figure, so hidden heterogeneity surfaces before the claim is trusted.
Independent corroboration: The frozen evidence defines Stratified Analysis Protocol as 'Splits an aggregate into pre-declared strata and compares each subgroup's pattern against the pooled figure, so hidden heterogeneity surfaces before the claim is trusted', so its operative form is Analysis, Modeling & Optimization.
Nearest alternative: Representation, Specification & Plan — Stratified Analysis Protocol includes features of a static representation, map, specification, schema, or prospective plan that externalizes information, but its defining operation is an analytical, modeling, inference, comparison, or optimization procedure that derives insight or a solution.
Review outcome: Independent reviewer agreement; medium confidence.
Origin Attribution¶
Primary origin: Statistics & Experimental Design
Origin pattern: Single lineage
Present-day reach: Multi-domain
Rationale: Predeclared subgroup comparison exposes masked heterogeneity.
Related originating lineages:
- Data Science & Analytics — Analytical pipelines implement strata.
- Mathematics — Mathematical modeling, proof, and abstract-structure practice supplies a parallel or contributing lineage for the mechanism's defining operation: splits an aggregate into pre-declared strata and compares each subgroup's pattern against the pooled figure, so hidden heterogeneity surfaces before the claim is trusted.
Review resolution: The blind reviewers agree that statistics_experimental_design is the primary origin and differ only on alternate origin disagreement, domain reach disagreement. I preserve every independently explained alternate from both records rather than imposing a numeric cap. I retain single_lineage because the combined evidence shows one traceable formative lineage. The broader reach of multi_domain records portability separately from historical provenance; encyclopedia_synthesis=false preserves the affirmative synthesis judgment where either reviewer identified one.
Review outcome: Reconciled after independent review; high confidence.
Notes¶
This protocol is deliberately upstream and diagnosis-agnostic: it builds the stratified table but takes no position on which level to trust. That verdict is where siblings such as Simpson's Paradox Check and Subgroup Dashboard with Warning Flags begin — both consume the tables this protocol produces.
[n1] The Cochran–Mantel–Haenszel procedure combines stratum-specific 2×2 associations into a single estimate while holding the stratifying variable fixed — the standard way to pool across strata without letting composition differences distort the summary. ↩