Candidate-Family Comparison Grid¶
Comparison — instantiates Distributional-Assumption Governance
Lays credible distribution families and assumption-light baselines side by side and scores them on support, rationale, tail behavior, and complexity so the family choice is argued, not defaulted.
A distribution chosen alone is a distribution never compared. Candidate-Family Comparison Grid forces a counterfactual onto the page: it lines up the preferred family against a set of credible rivals and at least one assumption-light baseline, and scores every one of them across the same fixed columns — support, substantive rationale, tail behavior, complexity, and how well each reproduces the data. Its job is to convert a silent default ("we used a normal") into an argued selection ("here are five families that could have generated this, and here is why one wins on the dimensions that matter"). The grid stops at the families' merits; it ranks and reasons about them, but it does not carry each choice all the way through to the final decision — it exposes where the choice is contested and hands the load-bearing disagreement onward. What it guarantees is that no family was adopted because it was the software's first offer.
Example¶
An insurer models claim severity for a property line before setting reserves. Instead of defaulting to a lognormal because it fits the middle, the actuary builds a grid. Rows: lognormal, gamma, a spliced body-plus-generalized-Pareto tail, and — as an assumption-light anchor — the raw empirical distribution. Columns: support (all four respect nonnegative claim sizes), mechanism rationale (a multiplicative accumulation of loss factors argues for lognormal; rare catastrophic claims argue for a heavy Pareto tail), tail behavior (lognormal and gamma both understate the largest claims that the empirical and Pareto tails capture), complexity (the splice adds a threshold parameter), and reproduction of the observed distribution.
The grid's payoff is a specific, visible disagreement: the family that wins the center-fit column (lognormal) loses the tail column, and the tail is exactly where large-claim reserving lives. That single crossing — a good average fit hiding an understated tail — is what the grid is for. It does not compute the reserve under each family; it flags that the reserve will differ and marks the tail column as the one to resolve, so the choice becomes a debate about the tail rather than an unexamined default.
How it works¶
- Enumerate credible rivals, not straw men. The candidate set must include families a knowledgeable skeptic would actually propose, plus an assumption-light baseline (empirical, nonparametric, or a simple bound) as a floor.
- Fixed columns, no single winner. Every candidate is scored on the same dimensions, and no one column — not even a formal fit score — is allowed to decide; a family that wins on fit but fails on support or mechanism is not accepted.
- Surface the flips. The grid's value is in the columns where the ranking changes, because those are the dimensions the decision is exposed to.
- Anchor with the assumption-light baseline. If the preferred family barely beats a nonparametric floor, the extra assumptions are not buying much.
Tuning parameters¶
- Candidate breadth — how many rival families and baselines to include. Wider grids resist convenient framing but cost effort and can dilute focus.
- Column set and weighting — which dimensions matter for this use, and how they trade off. Over-weighting fit reintroduces the default it was meant to prevent.
- Baseline stringency — how demanding the assumption-light comparator is. A tougher floor makes the case for a parametric family harder to win.
- Scoring rigor — qualitative judgment versus quantitative scores per cell. More rigor is comparable across reviewers but invites false precision.
When it helps, and when it misleads¶
Its strength is that it kills software-default commitment at the source: a family cannot be adopted without a named alternative that lost, and the tail and mechanism columns catch the family that fits the center while missing the region that drives the decision.
Its failure mode is information-criterion absolutism — collapsing the whole grid to a single number and letting, say, the Akaike Information Criterion crown a winner.[1] An information criterion trades fit against parameter count; it says nothing about whether the family respects the support, tells a plausible mechanistic story, or behaves in the tail that matters. The classic misuse is a rigged candidate set — three convenient families and no heavy-tailed rival — so the comparison is theater. The discipline that keeps it honest is to treat every column as a veto, not a vote: keep the mechanism and tail columns live, include a genuine assumption-light floor, and let the fit score be one column among several rather than the judge.
How it implements the components¶
alternative_family_and_assumption_light_comparator— the grid is the comparator: a structured set of credible rival families plus an assumption-light baseline evaluated against the preferred commitment.substantive_rationale_and_mechanism_link— the rationale column asks, for each candidate, why the underlying process could plausibly generate that shape, so a mathematically adequate but substantively impossible family is exposed.support_tail_zero_and_mixture_review— the support and tail columns check whether each family respects possible values and captures zeros, heavy tails, and mixtures.
The grid ranks families on their own merits but does not run each one through to the final decision output to measure how far the action actually moves (decision_consequential_sensitivity_map) — it flags where consequence would differ and hands that to the Distributional Sensitivity Grid.
Related¶
- Instantiates: Distributional-Assumption Governance — the grid supplies the "compare credible alternatives" step the rest of the lifecycle relies on.
- Consumes: Support, Shape, and Tail Diagnostic Suite — the empirical shape profile that each candidate is scored against.
- Sibling mechanisms: Distributional Sensitivity Grid · Support, Shape, and Tail Diagnostic Suite · Distributional-Assumption Card · Independent Assumption-Challenge Gate
Draft mechanism page for the Encyclopedia of Abstractions.
Editorial Notes¶
Form Classification¶
Form family: Analysis, Modeling & Optimization
Rationale: The mechanism scores credible distribution families and an assumption-light baseline on common dimensions to expose fit, support, tails, and complexity trade-offs, so its operative form is comparative analysis.
Nearest alternative: Decision, Gate & Allocation — The grid informs a family choice, but it produces the structured comparison rather than committing the final selection.
Review outcome: Adjudicated after independent review; high confidence.
Origin Attribution¶
Primary origin: Statistics & Experimental Design
Origin pattern: Single lineage
Present-day reach: Specialized
Rationale: Statistical model selection developed comparison of candidate distribution families on fit, support, tail behavior, assumptions, and penalized complexity.
Related originating lineages:
- Data Science & Analytics — Model-evaluation practice makes the comparison reproducible and includes assumption-light empirical baselines.
- Mathematics — Probability theory defines the candidate families and their support and tail properties.
Review resolution: Statistical model comparison is primary because the grid compares candidate families on common fit, assumption, and diagnostic criteria. Mathematical model families and data-science workflows are genuine formative lineages, while the method remains specialized and established.
Review outcome: Reconciled after independent review; high confidence.
References¶
[1] The Akaike Information Criterion (Akaike, 1974) ranks models by fit penalized for the number of parameters. It is one useful column in a comparison, but it cannot see support violations, implausible mechanisms, or tail failure — which is why collapsing the grid to a single AIC ranking reintroduces exactly the unexamined commitment the grid exists to prevent. withdrawn registry ↩