Segment Funnel Comparison¶
Comparative method — instantiates Funnel Attrition Localization
Re-runs the same funnel separately within meaningful slices — channel, device, region, cohort, access group — to reveal whether a whole-funnel drop is really one segment collapsing at one stage.
Segment Funnel Comparison takes a single funnel and computes it separately inside meaningful slices of the population — acquisition channel, device or platform, region, tenure cohort, user role, plan tier — then lays the stage-conversion profiles side by side to see where they diverge. Its defining idea is that an aggregate funnel is an average, and averages hide structure: a stage that looks mediocre overall may be fine for most people and catastrophic for one segment, or fine for everyone but dragged down in the blend by a shift in traffic mix. By splitting before comparing, it converts "this stage is weak" into the sharper, more actionable "this stage is weak for whom" — usually where the real cause and the real fix live.
Example¶
A news publisher sees its subscription signup funnel — landing → plan-select → account-create → payment → active — converting worse this quarter than last, with the softness spread thinly across every stage. The aggregate offers no handhold. The team re-runs the funnel by acquisition channel: organic search, paid social, newsletter referral, and a syndication partner. Split apart, the "spread-out" weakness resolves into one thing — the paid-social slice converts far worse than the others at exactly the account-create stage, while every other channel's profile is essentially unchanged from last quarter. The aggregate had drooped only because paid-social's share of traffic had grown.
That comparison relocates the problem entirely. It is not a general signup problem to fix everywhere; it is either a mismatch between what the paid-social ad promises and what account-create asks, or lower-intent traffic from that channel — both narrow, both testable. A second slice, by device, adds that the gap is worst on mobile, narrowing it further. None of this was visible in the blended funnel.
How it works¶
- Choose slicing dimensions that plausibly carry a cause — how people arrived, what they use, who they are — not just any field that happens to exist.
- Compute the full stage-by-stage conversion profile within each slice, on identical stage definitions, so the profiles are genuinely comparable.
- Compare at the stage where the gap between segments is largest, not where every segment is equally weak — a uniformly weak stage is a design problem; a segment-specific one is a targeting or fit problem.
- Watch for mix effects: a worsening aggregate can come entirely from a shift in segment proportions while every segment's own funnel is unchanged.
It localizes the loss to a who, not to a stage-in-isolation; that is its whole contribution.
Tuning parameters¶
- Slice dimension — which axis to cut on. The highest-leverage and hardest choice; the right dimension makes the gap leap out, the wrong one smears it back into the average.
- Slice granularity — a few broad segments or many fine ones. Finer slices localize better but thin each cell until differences are just noise.
- Comparison baseline — segments against each other, against their own prior period, or against the aggregate. Each answers a different "diverges from what?"
- Mix-adjustment — whether to hold segment proportions constant to separate "each segment got worse" from "the mix shifted toward weak segments." Skipping it invites the wrong diagnosis.
- Multiplicity discipline — how many slice/stage combinations you scan before believing a gap. Slice enough ways and a striking-looking gap appears somewhere by chance.
When it helps, and when it misleads¶
Its strength is defeating the average: it exposes Simpson's paradox cases, where an aggregate trend reverses or vanishes once the population is split, and it turns a diffuse "everything is a bit worse" into a located "this segment, this stage."[1] It is also the natural home of the equity lens — the same machinery, pointed at access and demographic slices, shows whether a stage quietly filters out a protected or under-served group.
Its failure mode is that slicing is fishing-prone: cut a funnel enough ways and some segment shows a dramatic gap by pure chance, and the tool is easily run backwards — keep re-slicing until a cut supports the story you already wanted, then present that slice as the finding. Thin cells make it worse, lending vivid but unstable numbers to tiny sub-populations. The discipline is to fix the hypothesis and the slice before looking, treat post-hoc gaps as leads to test rather than conclusions, and mix-adjust before blaming a segment for a change the traffic mix actually caused.
How it implements the components¶
Segment Funnel Comparison realizes the disaggregation side of the archetype — the components that ask not where the funnel leaks but for whom:
segment_or_cohort_slice— its core operation: partition the population by a cause-bearing dimension and compute the funnel within each part.fairness_and_access_slice— the same slicing pointed at access, demographic, and protected-group dimensions, surfacing differential drop-off that an aggregate hides.
It does not build the ordered picture or the per-stage loss magnitudes it re-slices (that is Stage Drop-Off Waterfall), rank which located loss to fix first (that is Loss Pareto Review), or verify that each segment's data captures its true population (that is Survivorship Bias Audit).
Related¶
- Instantiates: Funnel Attrition Localization — supplies the for-whom dimension the archetype needs to avoid optimizing a stage that is only weak on average.
- Consumes: Stage Drop-Off Waterfall or an equivalent stage-metric source, which it re-computes within each slice.
- Sibling mechanisms: Stage Drop-Off Waterfall · Loss Pareto Review · Stage Conversion Anomaly Alert · Survivorship Bias Audit · Cohort Transition Table · Conversion Funnel Dashboard · Denominator Reconciliation Checklist · Event Instrumentation Specification · Event Trace Process Mining · Funnel Experiment Backlog
Editorial Notes¶
Form Classification¶
Form family: Analysis, Modeling & Optimization
Rationale: Segment Funnel Comparison operates as an analytical, modeling, inference, comparison, or optimization procedure that derives insight or a solution because it re-runs the same funnel separately within meaningful slices — channel, device, region, cohort, access group — to reveal whether a whole-funnel drop is really one segment collapsing at one stage.
Independent corroboration: The frozen evidence defines Segment Funnel Comparison as 'Re-runs the same funnel separately within meaningful slices — channel, device, region, cohort, access group — to reveal whether a whole-funnel drop is really one segment collapsing at one stage', so its operative form is Analysis, Modeling & Optimization.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Data Science & Analytics
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Comparing staged conversion rates across segments is a data-analysis operation built from event funnels and stratified comparison. Google Analytics explicitly supports funnel reports and segment overlays; statistics governs uncertainty while behavioral and economic domains interpret the differences.
Related originating lineages:
- Behavioral Economics — behavioral_economics contributes decision bias, scarcity salience, mental accounting, and debiasing to this mechanism's defining operation—Re-runs the same funnel separately within meaningful slices — channel, device, region, cohort, access group — to reveal whether a whole-funnel drop is really one segment collapsing at one stage—without displacing the selected primary historical lineage.
- Economics & Finance — economics_finance contributes incentives, valuation, allocation, contracting, and opportunity cost to this mechanism's defining operation—Re-runs the same funnel separately within meaningful slices — channel, device, region, cohort, access group — to reveal whether a whole-funnel drop is really one segment collapsing at one stage—without displacing the selected primary historical lineage.
- Mathematics — Mathematical modeling, proof, and abstract-structure practice supplies a parallel or contributing lineage for the mechanism's defining operation: re-runs the same funnel separately within meaningful slices — channel, device, region, cohort, access group — to reveal whether a whole-funnel drop is really one segment collapsing….
- Organizational & Management Science — Service and marketing management independently interpret stage-specific segment collapse.
- Experimental Design & Statistics — Stratified analysis materially detects aggregation reversals.
Review resolution: The blind reviewers disagree on primary lineage (data_science versus statistics_experimental_design). Authoritative or primary research supports data_science as the best historical origin: Comparing staged conversion rates across segments is a data-analysis operation built from event funnels and stratified comparison. Google Analytics explicitly supports funnel reports and segment overlays; statistics governs uncertainty while behavioral and economic domains interpret the differences. The cited Google Analytics, Funnel Reports; Google Analytics, Apply Segments to Funnel Explorations directly supports the mechanism's defining operation. All independently supported contributing domains are retained without an arbitrary cap. origin_mode=cross_disciplinary_synthesis records the lineage relationship, while domain_reach=multi_domain records later applicability separately from provenance.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Researched adjudication after independent review; high confidence.
Sources consulted:
References¶
[1] Simpson, E. H. "The Interpretation of Interaction in Contingency Tables". Journal of the Royal Statistical Society: Series B (Methodological) 13(2), 238–241 (1951). Demonstrates that an association present within strata can disappear when the strata are combined. registry ↩ Show verification details
Supported in partVerified against the publisher's abstract
Simpson's paper shows by example that a stratified reading can differ from the combined table, the core of the paradox the sentence names.
“it is shown by an example that the vanishing of this second order interaction does not necessarily justify the mechanical procedure of forming the three component 2 × 2 tables and testing each of these for significance by standard methods.”