Pilot-and-Scale Feedback Review¶
Procedure — instantiates Top-Down / Bottom-Up Synthesis
Runs limited local implementations, examines what they reveal, and revises the central plan before wider rollout.
Pilot-and-scale feedback review is a procedure that runs a central plan in a small, deliberately representative set of local sites, records what the trial reveals, and adjudicates whether to revise the plan, hold, or scale — before wider rollout. Its defining idea is that it gates scaling on field evidence from a bounded, representative trial, and treats the central assumptions as genuinely revisable from what the pilot shows. Unlike a recurring cadence over ongoing work, it is a one-pass gate before commitment; unlike strategy formation, its test is empirical rather than deliberative. The whole point is to buy real-world evidence cheaply, on a small footprint, before the plan's cost becomes irreversible at scale.
Example¶
A restaurant chain wants to roll a new kitchen-display system to all 1,200 locations. Rather than a big-bang deployment, it pilots in a deliberately representative sample: a high-volume urban store, a drive-thru-heavy suburban one, a low-traffic small-town location, and a mix of franchise and corporate units. The pilot's traced findings show ticket times improving everywhere — except the drive-thru store, where the new screen layout hid modifier orders and slowed the line. The review applies its resolution rule against the central assumption ("this speeds every store"): the evidence contradicts it for one store type, so the verdict is revise — fix the drive-thru layout and add a training step — rather than scale as-is or abandon the system. Only after the revised plan clears does the rollout proceed. The rollout was gated on whether a representative trial actually confirmed the plan, store type by store type.
How it works¶
- Choose sites for representativeness. Select a sample that spans the variation that matters — volume, format, ownership — not just the convenient or friendly sites.
- Run the plan and trace the findings. Implement in the pilot sites and record what happens, linked to each site's context so surprises are attributable.
- Adjudicate against central assumptions. Apply an explicit rule — revise, hold, or scale — testing the evidence against what the plan assumed rather than against hope.
- Gate the rollout. Only a plan that survives the pilot verdict proceeds to wider scale; a failed assumption sends it back for revision first.
Tuning parameters¶
- Sample representativeness versus size — how many site types the pilot spans against its cost. Broad coverage catches failure modes but is slow and expensive; a thin sample is cheap but blind to the cases it omits.
- Pilot duration — how long the trial runs. Longer exposes seasonal and fatigue effects but delays the decision; shorter is fast but risks reading a honeymoon as steady state.
- Scaling threshold — how strong the evidence must be to green-light rollout. High is safe but stalls good plans; low ships fast but scales unverified bets.
- Revise-versus-abandon bar — how much contradiction triggers a redesign rather than a kill. Sensitive protects sunk effort but tolerates weak plans; strict is decisive but wasteful.
- Observation intensity — how closely the pilot is watched. Heavy observation yields rich evidence but distorts behavior; light is realistic but coarse.
When it helps, and when it misleads¶
Its strength is catching a plan's failure cheaply, in the field, before full commitment — the mismatch surfaces on four sites instead of twelve hundred. Its classic failure mode is the pilot that succeeds because of the attention and motivation of being the pilot, then fails at scale once that spotlight is gone: performance improves from being observed and special-cased rather than from the change itself.[n1] A second failure is the unrepresentative sample — friendly sites that flatter the plan and hide the cases where it breaks. The guarding discipline is to select sites for representativeness rather than convenience, and, where feasible, to run a quiet comparison site so the pilot's result can be checked against ordinary conditions rather than the glow of attention.
How it implements the components¶
representative_local_sample— the deliberately spanning set of pilot sites is the sample whose representativeness the whole verdict depends on.evidence_trace— the recorded, context-linked pilot findings are the evidence the gate decision rests on.conflict_resolution_rule— the explicit revise/hold/scale gate is the rule that adjudicates pilot evidence against the central assumptions.
Does not implement implementation_learning_cadence or feedback_loop — the recurring, cyclical reprioritization of ongoing work is Agile Portfolio Governance; nor central_intent / synthesis_forum, the formation-time apparatus of Participatory Strategy Process. This is a one-pass field gate before scaling — not a standing cadence and not a strategy-forming forum.
Related¶
- Instantiates: Top-Down / Bottom-Up Synthesis — supplies the bounded local trial whose evidence lets central assumptions be revised before rollout.
- Consumes: Frontline Feedback System instruments the pilot sites and captures the field signal the review examines.
- Sibling mechanisms: Participatory Strategy Process · Executive-Guided Co-Design · Frontline Feedback System · Federated Governance Cadence · Local-National Policy Synthesis · Agile Portfolio Governance · Community-Informed Implementation
Editorial Notes¶
Form Classification¶
Form family: Experiment, Test & Rehearsal
Rationale: Pilot-and-Scale Feedback Review operates as an active test, trial, simulation, drill, or rehearsal that generates evidence through a deliberate attempt or perturbation because it runs limited local implementations, examines what they reveal, and revises the central plan before wider rollout.
Independent corroboration: The frozen evidence defines Pilot-and-Scale Feedback Review as 'Runs limited local implementations, examines what they reveal, and revises the central plan before wider rollout', so its operative form is Experiment, Test & Rehearsal.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Organizational & Management Science
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Pilot-and-Scale Feedback Review is rooted in organizational and management science: Organizational learning uses pilot evidence and field feedback to revise a central implementation plan.
Related originating lineages:
- Public Administration & Policy — Public administration and policy materially shaped Pilot-and-Scale Feedback Review through policy implementation, regulatory administration, and public service. Pilot policy implementation and feedback from local administrators materially shaped top-down/bottom-up adaptation.
- Statistics & Experimental Design — Experimental design and statistics materially shaped Pilot-and-Scale Feedback Review through randomization, inference, sensitivity analysis, and validation.
Review resolution: Both blind reviewers agree that organizational and management practice is the primary origin. Reconciliation resolves alternate_origin_disagreement. Formative alternate lineages are retained as public_administration_policy, statistics_experimental_design; later breadth of use is recorded separately as domain_reach=multi_domain, while origin_mode=cross_disciplinary_synthesis describes the relationship among origin lineages.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Reconciled after independent review; medium confidence.
Notes¶
[n1] The Hawthorne effect names the tendency of people to change their behavior — usually to perform better — simply because they know they are being observed. It is the archetypal reason a pilot can outperform its own eventual rollout: the improvement rode on the attention, not the intervention. ↩