Frame Compatibility Review¶
Audit procedure — instantiates Conditioned Probability Frame Specification
A gate run before two probabilities are compared or pooled, checking that their events, denominators, time windows, and sampling rules are actually commensurable.
Two probabilities that look comparable often aren't. Frame Compatibility Review is the checkpoint you run between having two numbers and doing anything with them together — comparing, ranking, subtracting, or averaging. It does not produce a new probability; it renders a verdict on whether a proposed operation is legitimate, by interrogating whether the two claims share a target event, a denominator, a time window, and a sampling rule. Its defining move is that it is a gate on an operation, not a description of a number: it can pass one comparison and fail another for the very same pair, because compatibility is a property of the pair-plus-intended-use, not of either number alone.
Example¶
An insurer wants to know which of two regional books is riskier and pulls the loss rates: Book A at 6.2%, Book B at 4.9%. The obvious read is "B is safer." Frame Compatibility Review runs before that conclusion is allowed to stand. It asks: same target event? A counts any reported claim; B counts only claims that closed above the deductible — different numerators. Same denominator? A's rate is over policies-in-force at year-end; B's is over policies-written during the year — different populations. Same window? A is a full calendar year; B is a trailing nine months, so its losses haven't fully developed. The review's verdict: not comparable as stated, with a punch list of the three adjustments needed before the 6.2 and the 4.9 can sit in the same sentence.
The value delivered is a stopped mistake. Absent the review, a portfolio decision rides on a comparison that was never valid; with it, the two books are either re-cut onto a common frame or the comparison is abandoned.
How it works¶
The review walks a fixed checklist against the two (or more) claims:
- Target-event match — do the numerators count the same thing?
- Denominator match — is the population left after conditioning the same, or has it drifted between claims?
- Window and sampling match — same time horizon, same selection and censoring rules?
- Assumption overlap — are the two computed under assumption sets that can coexist, or does one exclude the other?
Each check yields pass / fail / repairable. A single hard fail blocks the operation; a repairable fail returns the specific transformation (re-cut the denominator, align the window, re-label the event) required to make the frames commensurable. The output is a decision plus, where possible, the repair — never a silent "close enough."
Tuning parameters¶
- Strictness threshold — how large a frame mismatch is tolerated before the comparison is blocked. Tight thresholds catch subtle apples-to-oranges errors but stall routine work with pedantic flags.
- Checklist depth — a three-line sanity check versus a full audit of event, denominator, window, sampling, and assumption. Depth buys rigor at the cost of speed.
- Repair-vs-reject stance — whether the review's default is to propose an adjustment or to refuse the comparison outright. Repair keeps analysis moving but risks over-massaging incompatible numbers into a false match.
- Scope of review — every comparison versus only high-stakes or cross-source ones. Universal review is safest but expensive; triage concentrates effort where a denominator error is costly.
When it helps, and when it misleads¶
Its strength is that it intercepts the single most common conditional-probability failure — comparing rates with different denominators — before the flawed comparison sets a decision. It is the mechanism that makes "apples to oranges" a checkable claim rather than a rhetorical complaint.
Its failure mode is that a review can manufacture comparability by aggressively repairing frames that should have stayed apart, producing a number that looks reconciled but averages away the very heterogeneity that mattered — the trap behind Simpson's paradox, where subgroup-honest rates reverse when force-pooled[1]. The classic misuse is running the review as a rubber stamp: ticking boxes to bless a comparison someone already wants to make. The guarding discipline is to treat a hard fail as a genuine stop — abandon or re-scope the comparison — and to keep the repair transformations visible so a reader can see what had to be assumed to make the two numbers meet.
How it implements the components¶
frame_compatibility_check— this component is the mechanism's core: the pass/fail verdict on whether two frames may be compared.assumption_boundary— the review makes each claim's assumption set explicit precisely to test whether the two can coexist.reuse_boundary— it enforces the boundary by refusing to let a number cross into a comparison outside its valid context.denominator_population— it inspects and reconciles the denominators as the primary axis of compatibility.
It does NOT implement base_rate_reference — surfacing the marginal rate beneath a set of subgroup rates is the stratified rate table's job; nor renormalization_rule or probability_type_label, which belong to the artifacts that compute and label numbers. The review adjudicates; it does not calculate.
Related¶
- Instantiates: Conditioned Probability Frame Specification — this mechanism enforces the compatibility invariant before comparison or reuse.
- Consumes: Reference Population Note and Conditional Probability Annotation supply the declared frames the review reads.
- Sibling mechanisms: Conditional Probability Annotation · Given-That Clause · Likelihood-Ratio Frame · Probability Tree · Reference Population Note · Scenario Condition Card · Stratified Rate Table · Two-by-Two Probability Table
Editorial Notes¶
Form Classification¶
Form family: Assessment, Review & Assurance
Rationale: The review evaluates whether events, denominators, time windows, and sampling rules make two probabilities commensurable and produces a compatibility finding.
Nearest alternative: Decision, Gate & Allocation — An incompatibility can block pooling, but the mechanism's defining output is the evidence-based frame finding.
Review outcome: Adjudicated after independent review; high confidence.
Origin Attribution¶
Primary origin: Statistics & Experimental Design
Origin pattern: Single lineage
Present-day reach: Universal
Rationale: Checking event definitions, denominators, time windows, and sampling frames before comparing probabilities is canonical statistical practice.
Related originating lineages:
- Data Science & Analytics — Dataset and metric governance operationalize compatibility checks before pooling dashboard or model outputs.
- Mathematics — Conditional probability formalizes why probabilities with different conditioning events are not directly commensurable.
Review resolution: Both reviewers agree that statistics_experimental_design is primary. I retain mathematics, data_science only as formative origin lineage(s), without treating every later application as an origin. single_lineage is appropriate because the evidence supports one principal professional lineage. Reach is universal as a separate applicability judgment: it does not widen or narrow the recorded provenance. Encyclopedia synthesis is true because the exact generalized artifact is an encyclopedia-authored combination or refinement. The secondary differences are reconciled with no unresolved primary-provenance ambiguity.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Reconciled after independent review; high confidence.
References¶
[1] Simpson, E. H. "The Interpretation of Interaction in Contingency Tables". Journal of the Royal Statistical Society: Series B (Methodological) 13(2), 238–241 (1951). Shows how associations within subgroups can reverse when the groups are combined. registry ↩