Rationale Coding Matrix¶
Artifact — instantiates Structured Expert Judgment Iteration
Organizes reasons, evidence types, assumptions, and counterarguments behind expert judgments across rounds.
A Rationale Coding Matrix is the structured artifact that captures why — the reasons, evidence types, assumptions, and counterarguments behind each judgment — and lays them out in a grid so the reasoning can be inspected the way the numbers already are. Its defining move is coding: each rationale is tagged by type (mechanistic argument, analogy, cited datum, untested assumption) and cross-tabulated against positions and rounds, so a reader can see not just that the group split but on what grounds, and can watch which arguments moved people between rounds. It is neither the elicitation nor the aggregate; it is the reasoning ledger. Crucially, it is built to hold a minority reason with the same fidelity as the majority's — the one dissenting argument gets a row, not a footnote — because the whole point is that a lone well-grounded objection should remain findable after the averaging is done.
Example¶
A product security team is rating the severity of a newly reported vulnerability, and the initial scores are all over the map — some engineers call it critical, others moderate. Rather than average the scores or let the most senior responder's call stand, they build a Rationale Coding Matrix alongside their severity scoring (framed against the standard CVSS metrics[n1]). Each engineer's rating gets its reasons entered and coded: "assumes default config exposes the port" (untested assumption), "similar to last year's CVE that was widely exploited" (analogy), "PoC exploit already circulating" (cited evidence), "requires local access most deployments block" (mitigating condition).
The matrix immediately reorganizes the disagreement. It shows the critical-vs-moderate split is not about the vulnerability's mechanics — everyone agrees on those — but about one contested assumption: whether the vulnerable port is exposed by default. The lone engineer arguing it usually is not gets a preserved row with her evidence, not a dismissal. The team's convergence-and-disagreement view now reads: consensus on impact, unresolved on exposure likelihood, with the minority rationale explicit. That is a far more decision-useful output than "average severity: high," and it tells them exactly what to check to resolve the split.
How it works¶
- Extract reasons. For each judgment, record the rationale, the evidence it rests on, and the assumptions it makes.
- Code by type. Tag each reason (evidence, analogy, mechanism, assumption, value, counterargument) so patterns across the panel become visible.
- Cross-tabulate. Lay rationales against positions and rounds, exposing which arguments cluster with which conclusions and which arguments actually shifted people.
- Preserve minorities intact. Give each distinct dissenting rationale its own durable entry so it survives aggregation rather than dissolving into it.
Tuning parameters¶
- Coding scheme granularity — a few broad reason-types or a fine taxonomy. Fine coding reveals subtle patterns but is laborious and invites coder disagreement about tags.
- Minority preservation threshold — how rare a rationale must be to still earn its own row. Preserving everything protects dissent but can bury the signal in idiosyncratic one-offs.
- Evidence-typing strictness — how sharply "cited datum" is distinguished from "assumption." Strict typing exposes ungrounded claims but slows entry and can feel adversarial.
- Round tracking depth — whether the matrix logs rationale changes per round or only the final state. Tracking changes reveals why views moved but multiplies the upkeep.
When it helps, and when it misleads¶
Its strength is turning a vote into an argument map: it distinguishes structured judgment from mere counting by making the grounds inspectable, and it is the archetype's main defense against a well-reasoned minority warning vanishing behind consensus language — the dissent stays on the page with its evidence attached.[n1] It also localizes disagreement to the specific assumption or datum in dispute, which is where resolution effort should go.
Its failure mode is coding that smuggles in bias: the person tagging rationales can flatten a nuanced argument into a dismissive "assumption" label, and an over-engineered scheme can bury the one decisive objection under a hundred trivial rows. The classic misuse is building the matrix after a conclusion to retro-justify it — a reasoning ledger written to defend a decision rather than to expose how it was reached. The guarding discipline is to code rationales in the experts' own terms (or let them self-tag), keep minority rows visible, and populate the matrix during the rounds, not after the verdict.
How it implements the components¶
rationale_trace— it is the reasoning ledger: reasons, evidence, and assumptions recorded and coded so later users can inspect why judgments held or changed.minority_rationale_preservation— it gives each distinct dissenting reason a durable, coded entry so it survives aggregation.convergence_disagreement_report— cross-tabulating rationales against positions yields a grounded view of where the panel agreed and precisely what the remaining splits rest on.
It records reasoning; it does not run or terminate the process. It does not chain the rounds or set the stop (iteration_round, stopping_rule are Delphi Study's), and it does not visualize the pooled numbers live (uncertainty_distribution rendering is Judgment Aggregation Dashboard's).
Related¶
- Instantiates: Structured Expert Judgment Iteration — it is the reasoning-trace artifact the archetype's rationale and dissent invariants depend on.
- Sibling mechanisms: Anonymous Survey Round · Calibrated Probability Elicitation · Delphi Study · Expert Elicitation Protocol · Judgment Aggregation Dashboard · Structured Forecasting Panel · Policy Expert Panel Process · Technical Consensus Round
Editorial Notes¶
Form Classification¶
Form family: Representation, Specification & Plan
Rationale: Rationale Coding Matrix operates as a static representation, map, specification, schema, or prospective plan that externalizes information because it organizes reasons, evidence types, assumptions, and counterarguments behind expert judgments across rounds.
Independent corroboration: The frozen evidence defines Rationale Coding Matrix as 'Organizes reasons, evidence types, assumptions, and counterarguments behind expert judgments across rounds', so its operative form is Representation, Specification & Plan.
Nearest alternative: Record, Log & Register — Rationale Coding Matrix includes features of a persistent ledger, log, register, or case record that preserves history and traceability, but its defining operation is a static representation, map, specification, schema, or prospective plan that externalizes information.
Review outcome: Independent reviewer agreement; medium confidence.
Origin Attribution¶
Primary origin: Futurism & Strategic Foresight
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Preserving and comparing expert reasons across iterative rounds is closest to Delphi and structured foresight. RAND describes Delphi as an iterative expert-elicitation method developed for forecasting under uncertainty.
Related originating lineages:
- Ethnography & Qualitative Methods — Qualitative coding and memoing supplied the classification of arguments, assumptions, and counterarguments.
- Organizational & Management Science — Structured expert judgment and deliberative decision processes materially shape repeated rounds.
- Security Studies & Intelligence Analysis — Analytic argument mapping supplied explicit preservation of minority rationales and contested assumptions.
- Statistics & Experimental Design — Structured expert elicitation supplied convergence and uncertainty reporting.
Review resolution: The blind reviewers disagreed on primary lineage. Light authoritative research resolves the defining form in favor of futurism_foresight: Preserving and comparing expert reasons across iterative rounds is closest to Delphi and structured foresight. RAND describes Delphi as an iterative expert-elicitation method developed for forecasting under uncertainty. The other materially formative traditions are retained as alternates; current breadth of use remains separate as domain_reach=multi_domain.
Attribution caveat: The named matrix is an encyclopedia synthesis over qualitative coding and expert elicitation. The matrix form integrates several expert-judgment and qualitative-analysis traditions.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Researched adjudication after independent review; medium confidence.
Sources consulted:
Notes¶
[n1] The Common Vulnerability Scoring System (CVSS) decomposes a severity score into named metrics (attack vector, complexity, impact, and so on), so that a rating carries its reasons rather than a bare number. A coding matrix generalizes that discipline to any expert judgment: the score is only as trustworthy as the coded rationale beneath it, and disagreements usually live in one specific metric. ↩a ↩b