Comparison & Evaluation

← Back to Mechanisms by Solution Family

Solutions that place alternatives, cases, or outcomes against shared criteria so differences become visible and judgments become defensible.

37 mechanisms across 5 solution archetypes in this solution family. A mechanism inherits the primary family of the archetype it instantiates; family is about the move the solution makes, not the domain where it originated.

Contrastive Differentiation

Clarify a concept, option, signal, or identity by making its differences from nearby alternatives explicit.

12 mechanisms · View full solution archetype

• A/B Comparison — Puts exactly two alternatives head-to-head under a single shared question so the difference that decides between them becomes actionable.
• Annotation and Callout Layer — Overlays explanatory markers directly on the material that point at the specific spots where two similar-looking items differ and say why the difference matters.
• Before/After Analysis — Distinguishes a changed state from its prior state by holding the earlier condition as a baseline and reading the difference the intervening change actually made.
• Concept Disambiguation Examples — Teaches the boundary of a concept with a curated set of positive examples, non-examples, and near misses, so a learner can recognize the category in cases they have never seen.
• Confusion Audit — Works backward from real mistakes — misclassifications, wrong substitutions, ambiguous reads — to find which distinctions are actually failing and route them for sharpening.
• Contrast Table — Lays a set of items out as rows against shared dimensions as columns so their distinguishing features can be read off side by side in a single neutral view.
• Decision Rubric with Distinguishing Criteria — Turns the differences that separate categories into explicit written criteria and cut-points, so many people classify the same borderline cases the same way.
• Differential Diagnosis — Enumerates the plausible explanations for a case in hand and eliminates among them by the features that would distinguish one from another.
• Near-Miss Case Pairing — Sets a correct case beside a nearly identical incorrect one that varies in a single decisive respect, so the one distinction separating them is impossible to miss.
• Product or Option Comparison Matrix — Scores the available options against the features, costs, risks, and fit conditions that actually matter to this decision, so a choice among many becomes defensible.
• Signal Highlighting — Marks the one signal or state change that matters so it stands out from surrounding noise and is not lost among everything else on view.
• Visual Contrast Encoding — Maps a difference onto a visual channel — size, weight, shape, orientation, position, or colour — so distinctions become perceptible at a glance, decoded through a legend.

Deviant Case Analysis

When a case violates what the comparison set led you to expect, analyze the violation as evidence for theory refinement rather than dismissing it as noise or treating it as a story by itself.

7 mechanisms · View full solution archetype

• Boundary Condition Revision Workshop — Uses the deviant case to revise scope conditions, typologies, theory clauses, or decision rules with reviewers or domain experts.
• Deviant Case Selection Protocol — Selects a case that violates an explicitly stated pattern while documenting the universe of comparison and selection rationale.
• Deviation Residual Table — Displays the predicted or expected pattern against the observed case evidence so the magnitude and type of deviation are explicit.
• Follow-Up Case Sampling Plan — Selects follow-up cases, audits, or monitoring triggers to test whether the revision prompted by a deviant case is isolated, subgroup-specific, or broadly general.
• Matched Case Pairing Matrix — Pairs the focal deviant case with expected-pattern cases that share key background conditions but differ in outcome or mechanism.
• Omitted Variable Probe — Searches for variables, mechanisms, constraints, or contextual features absent from the original explanatory frame.
• Within-Case Process-Tracing Memo — Documents event sequence, mechanism evidence, actors, constraints, turning points, and rival-process evidence inside the deviant case.

Fairness-Standard Selection and Reconciliation

Make fairness explicit and contestable: state who is comparable, which differences matter, what standard governs, how competing standards are prioritized, and when outcomes or evidence require revision.

6 mechanisms · View full solution archetype

• Distributional Impact and Tail Audit — Compares benefit, burden, access, and error across parties and distribution tails.
• Fairness-Metric and Exception Stress Test — Probes proxies, gaming, baseline shifts, temporal drift, and exception capture.
• Fairness-Standard Comparison Table — Applies candidate standards to the same decision and displays reasons, winners, burdens, conflicts, and uncertainty.
• Participatory Fairness Deliberation — Structures affected-party evidence and reasons around candidate standards, consequences, and conflicts.
• Relevant-Difference Challenge Test — Requires differential treatment to connect to purpose and survive proxy, counterexample, and necessity review.
• Rights-Floor and Sufficiency Gate — Rejects or revises rules breaching declared rights or minimum sufficiency before aggregate optimization.

Risk-Adjustment and Benchmark Selection

Before calling performance abnormal, inefficient, or skillful, choose a benchmark that matches the relevant risk exposure, opportunity set, time horizon, and information conditions.

7 mechanisms · View full solution archetype

• Alternative-Benchmark Sensitivity Grid — A grid comparing conclusions across plausible benchmarks, factor sets, horizons, or reference populations.
• Benchmark Attribution Report — A report that decomposes raw performance into benchmark return, exposure effect, residual effect, and unexplained noise.
• Case-Mix Risk Stratification Table — A table or dashboard that groups cases by baseline severity or exposure before comparing outcomes.
• Multi-Factor Performance Model — A model that estimates expected performance from multiple risk exposures and treats residual performance as candidate abnormal performance.
• Out-of-Sample Benchmark Validation — A validation step that checks whether the benchmark model works outside the fitting sample or original period.
• Pre-Registered Benchmark Policy — A protocol that fixes benchmark-selection rules before outcomes are evaluated.
• Style-, Sector-, or Case-Matched Benchmark — A benchmark constructed from comparators matched to style, sector, case mix, mandate, or exposure profile.

Symmetry-Based Fairness

Treat equivalent cases equivalently unless a relevant asymmetry justifies different treatment.

5 mechanisms · View full solution archetype

• Anti-Discrimination Check — Holds a case fixed and flips only a protected characteristic — race, sex, religion, disability, age — to see whether treatment moves; a targeted symmetry test for the markers the law and ethics forbid from counting.
• Consistency Audit — Measures decisions already made across reviewers, units, and time to surface unexplained variation — treatment that shifted when only the decider or the date changed, not the case.
• Equal-Treatment Checklist — A fixed set of questions a reviewer works through on every case — what treatment must stay invariant, what difference would justify departing, and what to record — so the same discipline lands on each decision as it is made.
• Policy Symmetry Test — Interrogates a rule as written or as coded — would it hand equivalent cases different treatment when only an irrelevant feature changes? — to catch asymmetry and smuggled values in the policy itself, before a single case is decided.
• Precedent Analysis — Lines a current case up against specific prior decided cases and asks whether it is relevantly like them — so a decision rests on 'we handled the last one this way and nothing relevant has changed,' or an explicit, defensible distinction.