Skip to content

Small Safe-to-Fail Probe

Method — instantiates Catalytic Pathway Enablement

A deliberately small, contained trial that tests whether a proposed facilitator really lowers the barrier — and preserves selectivity — before it is trusted at scale.

Before a facilitator is switched on for every case, one question decides everything: does it actually lower the intended barrier without wrecking selectivity or importing side effects? Small Safe-to-Fail Probe answers it empirically and cheaply. It runs the proposed facilitator on a small, bounded slice of real substrate, sized so that failure is affordable and contained, and compares the result against a no-facilitator baseline. Its defining discipline is the safe-to-fail bound: the trial is designed so a bad outcome teaches something and costs little, rather than a large bet wearing a probe's clothing. It is the reality check between "this facilitator should work" and "deploy it everywhere."

Example

A city wants a facilitator that speeds low-risk residential permits: a pre-checked "instant-approve" pathway for a narrow class of rooftop-solar installs. Rather than switch it on citywide, the buildings department runs a safe-to-fail probe in one district for about eight weeks, capped at a few hundred permits, with a one-click rollback.

The probe watches three things. Did the barrier actually fall — permit cycle time on the new pathway versus the old process? Is the gain real — measured against a baseline of comparable permits in a neighboring district still running the old way, to rule out a seasonal lull? And did selectivity hold — did any unsafe or out-of-class application slip through the fast lane? Setup to outcome: cycle time drops from roughly six weeks to about four days on eligible cases; the baseline district confirms the gain is the facilitator and not the season; and an audit finds two out-of-class applications that leaked through — a selectivity gap the team closes before any citywide rollout. Because the probe was small and reversible, that gap cost almost nothing to discover.

How it works

  • Pick a small, representative slice and cap the blast radius so failure is genuinely safe — a real rollback, a bounded case count, contained harm.
  • Run facilitator-on against facilitator-off. A concurrent or matched baseline makes the effect attributable to the facilitator rather than to trend or selection.
  • Check feasibility and selectivity, not just speed. Confirm the target outcome was already achievable (so a win is real acceleration, not a redefined goal), and instrument for side-pathway and selectivity leaks that only appear under real cases.

Tuning parameters

  • Probe size / blast radius — bigger probes give tighter evidence but raise the cost and harm of failure; the safe-to-fail ceiling is the dial that keeps a test from becoming a bet.
  • Baseline rigor — from a simple before/after to a matched concurrent control. Stronger baselines rule out "it would have improved anyway," at more setup cost.
  • Slice representativeness — a clean, easy slice validates fast but flatters the facilitator; a hard, representative slice is a truer test and far likelier to surface selectivity leaks.
  • Pre-committed pass / kill criteria — how much barrier reduction counts as success, and what selectivity leak or side effect triggers an automatic stop. Setting them in advance is what stops the probe being read backwards into a "yes."
  • Duration — long enough to see turnover and early fouling, short enough to stay contained.

When it helps, and when it misleads

Its strength is turning "should lower the barrier" into measured evidence, catching selectivity and side-path problems while they are still cheap, and — because it is bounded and reversible — carrying real option value: the freedom to stop is itself worth something.

Its failure modes are about representativeness and attribution. An unrepresentative or too-easy slice produces a rosy result that does not survive scale; a weak baseline lets ordinary improvement masquerade as catalytic effect; a probe watched only for speed misses the byproducts that appear at volume. The classic misuse is to run the probe after the decision is effectively made, then read its numbers backwards to ratify a rollout that was never really in doubt. A real anchor names the intended stance: the safe-to-fail probe of the Cynefin framework — in a complex domain you probe with contained experiments and amplify or dampen based on what you learn, rather than analyzing to certainty first.[1] The discipline is to pre-commit pass/kill criteria and a matched baseline, and to design the slice so it is actually able to fail.

How it implements the components

Small Safe-to-Fail Probe fills the empirical-validation components — the ones that test a claim rather than assume it:

  • activation_barrier_model — it puts the barrier-reduction claim to the test, measuring whether the specific barrier the facilitator targets actually falls in real cases.
  • baseline_and_counterfactual_measure — its core move is the with-versus-without comparison against a credible baseline, so any gain is attributable to the facilitator and not to trend or selection.
  • equilibrium_neutrality_check — it confirms the target outcome was already feasible, so a positive result is genuine acceleration rather than a mislabeled change in what is possible.

It does not size the barrier before the fact (Barrier Height Estimation estimates that up front), define the interface the facilitator uses (that's Interface Contract Design), or monitor capacity once the facilitator is live at scale (that's Active-Site Capacity Dashboard).

Notes

Safe-to-fail is a property of the probe's design, not its intent. A trial is only safe-to-fail if the rollback is real and the blast radius is capped before it starts. A pilot you cannot actually stop, or whose failure quietly harms real cases, is a scaled deployment wearing a probe's label — and it fails exactly where a probe is supposed to protect you.

References

[1] In Dave Snowden's Cynefin framework, complex problems call for safe-to-fail probes — small, contained experiments run in parallel, with pre-set signals to amplify what works and dampen what does not — rather than up-front analysis to certainty. ↩