Skip to content

Prototype Tournament

Prototype-driven selection process — instantiates Divergence-Convergence Cycle Orchestration

Builds competing prototypes and pits them against each other on tests deliberately designed to expose their differences, letting evidence rather than advocacy eliminate options.

When rival concepts can be argued about forever because everyone's spec sheet is optimistic, Prototype Tournament stops the debate by building the answer. It commissions a working prototype of each serious option family and runs them head-to-head on a task chosen for one property above all: it must discriminate — produce different results for different concepts — rather than merely show that each "works." Its defining move is to manufacture the evidence the decision needs instead of extracting it from opinion, so options are retired by what they did on the bench, not by who argued hardest for them. The prize isn't just a winner; it's a winner that arrives with a measured, public failure profile, which tells the cycle exactly what to de-risk next.

Example

A warehouse-automation team must pick a gripper approach for a pick-and-place robot: vacuum suction, a two-finger pinch, or a granular-jamming "coffee-bag" gripper. On paper each vendor claims to handle everything. Rather than adjudicate the claims, the team builds three rough grippers and designs a single deciding task — a bin deliberately mixed with porous cardboard, a heavy irregular casting, and a floppy poly-bag — precisely because the three approaches should diverge on it. They do: suction can't hold the porous item, the pinch drops the heavy irregular, and jamming grabs everything but slowly. The task was engineered to split the field, not to flatter any entrant, so the evidence retires two options cleanly. The survivor's weakness — cycle time — isn't a surprise buried for later; it's the measured result that becomes the next sprint's target.

How it works

  • Design the deciding test first. Before anything is built, name the one question whose answer separates the option families; the test exists to answer that, not to demonstrate general feasibility.
  • Build only enough to expose the difference. Prototypes are as rough as they can be while still revealing the distinguishing behaviour — realism is spent only where it changes the verdict.
  • Run under matched conditions. Entrants face the same task, inputs, and instrumentation so a win is attributable to the design rather than to a friendlier setup.
  • Retire on evidence, keep the raw results. Losing concepts are eliminated by measured performance, and the underlying data is retained so the decision — and any later reopening — can be re-examined.

Tuning parameters

The dials that adapt this tournament to a specific decision:

  • Discrimination target — a sharp decisive-question test (cheap, splits the field, but narrow) versus a broad benchmark suite (thorough, but slow and often fails to separate the contenders).
  • Prototype fidelity — throwaway rigs that just expose the difference versus near-production builds that cost more but settle more.
  • Bracket structure — round-robin (every concept meets the deciding test) versus elimination rounds (cheaper, but can drop a late-blooming option before its strength shows).
  • Matched-condition rigor — how tightly the test harness is controlled; looser setups are faster but let confounds masquerade as design differences.
  • Re-entry rule — whether an eliminated concept may return with a targeted fix, or whether elimination is final for this cycle.

When it helps, and when it misleads

Its strength is converting advocacy into evidence: differences that specs hide show up on the bench, and the process is the archetype's answer to "the loudest voice wins." It is worth its cost when the concepts genuinely might behave differently and the difference matters.

Its failure modes cluster around the test itself. A performative prototype demonstrates effort without probing the assumption that actually separates the options, so everyone passes and nothing is learned. Tournaments are expensive, and they are easy to rig — design the deciding task around the favourite's strengths, or against a rival's known weakness, and the "evidence" merely ratifies a decision already made. Elimination can also cut a slow-but-fixable concept whose problem was cheaper to solve than the winner's. The discipline that keeps it honest is to specify the discriminating question before building, to design the test to threaten the front-runner rather than to confirm it,[1] and to retain the raw results so the verdict is inspectable rather than asserted.

How it implements the components

Prototype Tournament fills the evidence-generating part of the convergence machinery — it produces the facts, not the framing:

  • evidence_and_prototype_plan — its core output: a plan of what to build and which test will discriminate among the option families, then the evidence that test yields.
  • convergence_criterion_set — entrants are judged against declared criteria under matched, real-world-ish conditions, so the criteria are exercised on behaviour rather than on claims.

It does not compare options on existing information — that paper judgment is Pairwise Option Comparison — and it does not decide how much to invest or when to fund the winner, which is Stage-Gate Concept Review. It also does not generate the competing families; it consumes them from Parallel Concept Sprints.

References

[1] Designing experiments specifically to distinguish between competing hypotheses — rather than to accumulate support for a favoured one — is the essence of strong inference, articulated by John Platt. A prototype tournament is that discipline in hardware: the deciding test earns its place only if the option families are expected to fail it differently. ↩