Delphi Study¶
Method — instantiates Structured Expert Judgment Iteration
Implements structured expert judgment through anonymous rounds, controlled feedback, and revision until useful convergence or stable disagreement is reached.
A Delphi Study is the multi-round engine of the archetype: it takes the single blind survey pass and chains it, feeding each round's anonymized result into the next so experts can revise in light of the group's reasoning, and it decides when the chain has run its course. Its defining feature is the loop plus its terminator — round two exists to let a judgment move after seeing round one, and a stopping rule exists so the loop ends when movement stops rather than when everyone caves. The method's central claim is that iteration under anonymity converts a scatter of first impressions into something a decision can lean on: either a convergence that survived scrutiny, or a disagreement that persisted despite it — and the study reports both as legitimate outputs.
Example¶
A specialty medical society must set a threshold: at what estimated risk level should clinicians recommend a particular preventive procedure, given that the trial evidence stops short of the exact population in question? Rather than let the loudest professor's number become the guideline, the society runs a Delphi Study over three rounds. In round one, thirty clinicians independently rate appropriateness across a grid of patient scenarios. The controlled feedback shows each clinician where their ratings sat relative to the anonymized distribution, plus the reasons dissenters gave. Round two lets everyone revise; several ratings shift as an overlooked comorbidity argument circulates, and the spread narrows on the clear-cut scenarios.
The study's discipline shows in what happens next. On the borderline scenarios, the spread does not narrow — a genuine split persists between clinicians weighting harms differently. The stopping rule (movement below a set threshold, and no new arguments appearing) ends the study at round three rather than pushing a fourth round to manufacture agreement. The final report marks the settled scenarios as consensus and flags the borderline ones as durable disagreement requiring a judgment call, not a hidden average.
How it works¶
- Chain the rounds. Run a blind elicitation, feed back the anonymized distribution and rationales, and re-elicit — each round seeded by the last, so revision is informed rather than blind.
- Control the feedback. Between rounds the method curates what comes back: the distribution, the reasons for outliers, sometimes new evidence — deliberately, not as a raw dump.
- Watch for a stop. Track how much estimates move round-to-round and whether new arguments still appear; when both fall below threshold (or a deadline hits), stop.
- Report both outcomes. Distinguish where judgments converged from where they held apart, and present persistent disagreement as a finding, not a failure.
Tuning parameters¶
- Round count / stopping threshold — how little movement counts as "settled." A loose threshold ends fast but risks stopping before learning is done; a tight one risks fatiguing experts into false convergence.
- Feedback content — statistics only, or statistics plus anonymized rationales and counterarguments. Richer feedback drives more genuine revision but demands heavy synthesis between rounds.
- Convergence definition — what the study counts as "consensus" (interquartile range, percent agreement). Set it too eagerly and stable disagreement gets papered over.
- Panel persistence — whether the same experts must complete every round. Attrition biases later rounds toward whoever stayed, which may not be a random subset.
When it helps, and when it misleads¶
Its strength is that it is the full loop: it lets judgments change on the merits under cover of anonymity, and it legitimizes "we still disagree" as a real answer — which is exactly what protects it from the committee-room failure of debating until objection is exhausted.[n1] Decades of use in foresight and clinical guideline work rest on that loop.
Its signature failure mode is false convergence driven by the stopping rule itself: run enough rounds and people tire, defer, and drift together, so the study reports agreement that is really fatigue. The classic misuse is a sponsor who treats consensus as the required deliverable and keeps the rounds going until the holdouts yield. The guarding discipline is a stopping rule tied to diminishing movement and no new arguments rather than to reaching agreement, plus an explicit license — stated up front — for the study to end in mapped disagreement.
How it implements the components¶
iteration_round— the chained revise-after-feedback loop is the method's core; each round exists so judgments can move in light of the last.stopping_rule— it decides when to end, on stability and diminishing returns rather than on achieved consensus.convergence_disagreement_report— it reports where views settled and where they durably split, treating persistent disagreement as a finding.
It orchestrates rounds but delegates the round mechanics: the blind capture and anonymized playback of any single pass are Anonymous Survey Round's (independent_elicitation, anonymized_feedback), and it does not itself select or COI-screen the panel (expert_panel, conflict_of_interest_screen) — that is Policy Expert Panel Process.
Related¶
- Instantiates: Structured Expert Judgment Iteration — Delphi is the archetype's best-known full realization.
- Consumes: Anonymous Survey Round supplies the single-pass blind round that the study repeats.
- Sibling mechanisms: Calibrated Probability Elicitation · Expert Elicitation Protocol · Judgment Aggregation Dashboard · Rationale Coding Matrix · Structured Forecasting Panel · Policy Expert Panel Process · Technical Consensus Round
Editorial Notes¶
Form Classification¶
Form family: Communication, Facilitation & Learning
Rationale: Delphi Study operates as a designed message, facilitated interaction, ritual, or learning activity that changes shared understanding because it implements structured expert judgment through anonymous rounds, controlled feedback, and revision until useful convergence or stable disagreement is reached.
Independent corroboration: The frozen evidence defines Delphi Study as 'Implements structured expert judgment through anonymous rounds, controlled feedback, and revision until useful convergence or stable disagreement is reached', so its operative form is Communication, Facilitation & Learning.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Futurism & Strategic Foresight
Origin pattern: Single lineage
Present-day reach: Multi-domain
Rationale: RAND forecasting research cohered Delphi studies around anonymity, iteration, controlled feedback, and an explicit stopping rule.
Related originating lineages:
- Operations Research — Structured expert judgment supplied formal aggregation and convergence criteria.
Review resolution: RAND forecasting research cohered Delphi studies around anonymity, iteration, controlled feedback, and an explicit stopping rule. Structured expert-judgment aggregation at RAND is a genuine operations-research lineage within the forecasting method.
Review outcome: Reconciled after independent review; high confidence.
Notes¶
[n1] The Delphi method was developed at the RAND Corporation in the 1950s–60s (Dalkey and Helmer) precisely to get useful group judgment without face-to-face domination; its defining features are anonymity, iteration, and controlled feedback. A "Delphi panel" lacking those — a one-shot survey, or open debate — is Delphi in name only. ↩