Lifetime Distribution Comparison¶

Model comparison — instantiates Survival-Conditioned Persistence Forecasting

Fits and pits rival lifetime distributions against each other to expose how much the remaining-life forecast hangs on which tail you choose to believe.

Once the hazard's shape is known, someone still has to say how much longer the thing will last — and that number is hostage to a modeling choice most forecasts make silently. Lifetime Distribution Comparison drags that choice into the open by fitting several candidate lifetime distributions to the same data and reporting how far the remaining-life forecast moves as you switch between them. Its defining move is to refuse to collapse model uncertainty into a single tidy answer: an exponential says remaining life is constant regardless of age, a Weibull with rising hazard says it shrinks, a heavy-tailed Pareto says it grows with age — and these can all fit the observed body of the data comparably while disagreeing by years about the future. The mechanism's product is that disagreement, made explicit, plus the recommendation of which evidence would actually separate the families.

Example¶

A people-analytics team wants the expected remaining tenure of an employee who has already stayed five years, to plan succession depth. They fit three families to the attrition data. An exponential (memoryless) says five years served tells you nothing — remaining tenure is the same as on day one. A Weibull with shape > 1 (rising hazard) says the veteran is, if anything, closer to a move. A lognormal / heavy-tailed fit says the opposite — that long-tenured staff are disproportionately "lifers" whose remaining tenure grows with service. All three match the body of the data at nearly the same information criterion, yet at the five-year mark they imply remaining tenures that are flat, shrinking, and growing respectively.

So the comparison does not hand back one figure. It hands back the fan of forecasts, the finding that the succession decision hinges entirely on the tail no one has enough data to pin down, and a confidence band that spans the families rather than hiding inside one. Its concrete recommendation is evidential, not numerical: gather the tail — do ten-year veterans actually leave at lower rates? — because that, not another fit statistic, is what would collapse the fan.

How it works¶

The comparison treats "which distribution" as an explicit, quantified decision rather than a default. What distinguishes it:

Fit rivals, not a favorite — several families (exponential, Weibull, lognormal, gamma, Pareto/power-law) are fit to the same censoring-corrected data.
Judge where the decision lives — they are compared not only by overall likelihood (AIC/BIC) but by fit in the decision-relevant region, using Q-Q and mean-residual-life plots that expose the tail.
Propagate the disagreement — the spread of remaining-life forecasts across the plausible families becomes the reported band; the model choice is carried as uncertainty, not resolved by fiat.

It does not read raw hazard shape (that is upstream) or tabulate empirical remaining life age-by-age; it prices how sensitive the forecast is to the distributional assumption.

Tuning parameters¶

Candidate slate — which families are on the table. Wider slates surface more model risk but dilute focus.
Fit-region weighting — whether selection optimizes fit to the body or the tail. The decision-relevant region — usually the tail — should drive the choice, not overall likelihood.
Selection criterion — AIC vs BIC vs out-of-sample. BIC penalizes complexity harder; an out-of-sample test scores forecasting rather than fit.
Averaging vs single pick — keep a weighted ensemble (an honest band) or commit to one family (sharper, riskier).
Band-width policy — how much of the cross-family disagreement is carried into the reported confidence band versus suppressed.

When it helps, and when it misleads¶

Its strength is that it exposes the thing the archetype most wants surfaced: a survival-conditioned remaining-life estimate can be dominated by an unfalsified tail assumption, and two models that explain the past equally can disagree by years about the future. It converts "the model says X" into "the answer swings from A to B depending on the tail, and here is what would tell them apart."

Its failure modes cluster around the tail. Best-fit is not best-forecast — the family that fits the observed body best can be worst exactly where the decision lives, and with few long-lived cases the tail is where data are thinnest, so overfitting a heavy tail to a handful of famous old survivors manufactures a Pareto "Lindy" result from noise. Choosing the family that yields the desired horizon is the backwards-run misuse. The clarifying anchor is the exponential's memorylessness — the null in which survival tells you nothing about remaining life, against which any claimed aging or Lindy signal must earn its keep.^[1] The discipline is to weight fit toward the decision-relevant region, keep the cross-family band, and treat the tail index as an assumption to stress rather than a fact to read off.

How it implements the components¶

This mechanism fills the estimate-and-its-uncertainty components on the modeling side:

conditional_remaining_life_estimate — it produces the remaining-life-given-survival estimate and shows how far that estimate travels as the distribution family changes; the sensitivity is the payload, not a footnote.
uncertainty_and_confidence_band — the spread of forecasts across plausible families is reported as the band, so model uncertainty (not merely sampling error) is made visible.

It does not read hazard shape non-parametrically (Hazard-Shape Diagnostic) or tabulate empirical remaining life age-by-age (Age-Conditioned Remaining-Life Table produces that actuarial table); it consumes the clean record and shape reading rather than producing them, and it does not set the planning horizon (Lindy Decision-Horizon Review).

Instantiates: Survival-Conditioned Persistence Forecasting — it supplies the remaining-life forecast and, crucially, the honest band around it.
Consumes: Hazard-Shape Diagnostic narrows the plausible families; Censoring and Left-Truncation Audit supplies the corrected risk set to fit against.
Sibling mechanisms: Hazard-Shape Diagnostic · Age-Conditioned Remaining-Life Table · Survival or Time-to-Event Analysis · Censoring and Left-Truncation Audit · Reference-Class Forecasting · Non-Aging Eligibility Review · Stationarity Test · Change-Point Detection · Rolling Window Comparison · Forecast Backtesting · Historical or Holdout Coverage Backtest · Periodic Durability Inspection · Lindy Decision-Horizon Review

Notes¶

A censoring- or survivorship-biased sample does not merely add noise to this comparison — it systematically favors heavy-tailed, Lindy-flattering families, because the missing failures are exactly the short lifetimes that would have anchored a lighter tail. That makes the Censoring and Left-Truncation Audit a precondition rather than a courtesy: run the comparison on an unaudited survivor set and the "best" distribution will lean durable for reasons that have nothing to do with durability.

References¶

[1] The exponential distribution is memoryless: its hazard is constant, so the expected remaining lifetime is the same at every age — survival to date carries no information about what remains. It is the natural null hypothesis here, the boundary between aging (remaining life falls with age) and Lindy-like durability (remaining life rises with age); a claimed signal in either direction is a claim that the lifetime is not exponential. ↩