Evidence Comparison Matrix¶
Artifact — instantiates Dissonance Resolution Pathway
Lays the competing claims, the evidence for and against each, and the alternative interpretations side by side on one surface, so a group argues from the display instead of from memory and defensiveness.
An Evidence Comparison Matrix is a static artifact — a table or grid — that externalizes a contested question. Its axes hold the competing claims and candidate interpretations; its cells hold the evidence for and against each, tagged with strength and gaps, so everyone inspects the same visible layout instead of arguing from selective recollection. Its defining feature — against a private belief-update protocol or a live workshop — is that it is a durable display whose whole value is making the structure of a disagreement visible at a glance. It organizes the evidence and the options; it deliberately stops short of naming the conflict as a protocol step, weighing the stakes, or committing a revision.
Example¶
A subscription app is losing users, and the team is split. One camp is sure "churn is an onboarding problem"; the other is sure "churn is a pricing problem." In meetings the two sides talk past each other, each citing whichever numbers it remembers. So they build an evidence comparison matrix. Two columns for the two hypotheses; a row for each piece of evidence — cohort retention curves, cancellation-survey verbatims, the results of a recent price-change A/B test, recurring support-ticket themes. Each cell marks whether that evidence supports, undercuts, or is silent on the hypothesis, with a short note on how strong it is. Laid out, the picture is not what either camp expected: the onboarding evidence is strong for early first-week drop-off but silent on month-three churn, exactly where the pricing evidence is strong. The matrix declares no winner — it makes visible that the two hypotheses explain different segments, the very thing that arguing from memory kept hiding. What the team does next is another step's job.
How it works¶
Its discipline is in how the grid is built and read, which is what separates it from a pro/con list:
- Fix the competing claims as the axes — the hypotheses or interpretations become the columns the evidence is scored against.
- Enter each piece of evidence once, marked for / against / silent, with a note on strength and known gaps.
- Make missing and ambiguous evidence visible — empty and weak cells are information, not blanks to skip.
- Read the pattern across the whole grid, not cell by cell — the shape of support and silence is the point.
Tuning parameters¶
- Claim granularity — a few broad hypotheses versus many narrow ones; finer claims localize the evidence but multiply the cells and the upkeep.
- Evidence-strength scheme — simple marks versus weighted or scored cells; scoring adds rigor but invites false precision.
- Treatment of silent cells — whether "no evidence either way" is recorded explicitly; making silence visible is often the matrix's biggest payoff.
- Authorship — a single compiler versus adversarial pairs filling opposing columns; adversarial authorship resists cherry-picking.
- Freeze vs. living — a snapshot for one decision versus a document updated as evidence arrives.
When it helps, and when it misleads¶
Its strength is defeating argument-from-memory: once the evidence is on one surface, one-sided support and conspicuous gaps become impossible to wave away, and a group can finally see where it actually disagrees. It is at its best when the disagreement is about facts people each half-remember.
Its central failure mode is that the matrix can be filled to confirm a favored hypothesis — selective entry, generous scoring of friendly evidence — and the tidy grid then lends false rigor to a foregone conclusion. A related misuse is treating the completed matrix as a decision rather than an input; a display of evidence is not a choice. The guarding discipline, borrowed from Analysis of Competing Hypotheses,[n1] is to enter disconfirming evidence deliberately and to look for the hypothesis with the least evidence against it — and to have someone whose job is to break the favored column.
How it implements the components¶
Evidence Comparison Matrix fills the externalized-evidence slice of the pathway:
evidence_review— it is the review, made visible: the artifact that arrays each claim against the evidence for and against it, with strength and gaps on the same surface.interpretation_options— its axes hold the competing interpretations side by side, keeping the alternatives explicit rather than letting the group collapse to the most comfortable single story.
It does not name the conflicting_belief_pair as a procedural step, calibrate stakes_and_reversibility_assessment, or commit the revision_choice — a matrix informs a revision, it does not make one. Converting the displayed evidence into a chosen, uncertainty-tagged update is Belief-Update Protocol's work, its nearest twin.
Related¶
- Instantiates: Dissonance Resolution Pathway — Evidence Comparison Matrix supplies the externalized evidence review the rest of the pathway reasons from.
- Sibling mechanisms: Accountability Conversation · After-Action Learning · Belief-Update Protocol · Commitment Rewrite · Facilitated Sensemaking Session · Restorative Dialogue · Reflective Dialogue · Values Clarification · Assumption Review
Editorial Notes¶
Form Classification¶
Form family: Representation, Specification & Plan
Rationale: Evidence Comparison Matrix operates as a non-executable information artifact that externalizes static or prospective structure because it lays the competing claims, the evidence for and against each, and the alternative interpretations side by side on one surface, so a group argues from the display instead of from memory and defensiveness.
Independent corroboration: The frozen evidence defines Evidence Comparison Matrix as 'Lays the competing claims, the evidence for and against each, and the alternative interpretations side by side on one surface, so a group argues from the display instead of from memory and defensiveness', so its operative form is Representation, Specification & Plan.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Security Studies & Intelligence Analysis
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Intelligence analysis formalized Analysis of Competing Hypotheses as a matrix that arrays evidence against rival explanations and emphasizes inconsistency and diagnosticity.
Related originating lineages:
- Operations Research — Side-by-side evidence and alternative matrices descend from decision-analysis comparison tables and multicriteria decision support.
- Organizational & Management Science — Facilitated organizational deliberation materially shaped the use of shared comparison surfaces to depersonalize disagreement.
- Statistics & Experimental Design — Systematic evidence comparison and explicit strength assessment descend from research-synthesis and comparative evaluation methods.
Review resolution: The CIA tradecraft primer specifies essentially the same hypothesis-by-evidence comparison surface. Statistical comparison and facilitated decision practice are formative additions to the encyclopedia's generalized group version.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Researched adjudication after independent review; high confidence.
Sources consulted:
Notes¶
[n1] Analysis of Competing Hypotheses — Richards J. Heuer, Jr.'s CIA tradecraft method: list all the hypotheses, array the evidence against each, and seek the hypothesis with the least inconsistent evidence rather than the most supporting evidence. The evidence comparison matrix is a general-purpose relative of this artifact. ↩