# Project 02 — Verb-engine design proposal (2026-05-25)

**Status: APPROVED + EXECUTED 2026-05-25. Decisions locked (below); apparatus built and the A/B/C ablation run. Results in `RESULTS.md` (headline: A≈B≈C on quality once semantic retrieval is live; free planner self-disciplines and is most robust + cheapest; mandatory enforcement is double-edged via over-build; the §9.2 erosion fear did not occur).**

**Locked decisions (the four forks):**
1. Planner realization → **structured-state loop** (explicit `{verb,args,rationale}`, explicit STATE, logged ops).
2. C's coverage-check → **separate independent verifier pass** (enforced, not self-asserted).
3. Scenarios → **~5, stratified by catalog maturity × obliqueness**, incl. ≥1 oblique/buried case.
4. Grading → **2 independent blinded graders + 2 reps/cell**.

This document proposes the three things the workflow-02 spec asks me to settle before building:
(1) the **state representation** the planner acts on, (2) the **verb library** with pre/post-conditions
(especially the lift's coverage check), and (3) the **planner mechanism** and the A/B/C ablation that
tests it. It closes with the **open decisions** that genuinely fork the work and a **repeatability** plan.

It is grounded in what Protocols 0–4 settled, restated once so the design is legible on its own:

- The lift is **not** typed by framed/structural. That gate was tested on 13 primes and refuted:
  `conservation_laws` (pure structural) got the biggest effect (+3.0); `symmetry` (pure structural)
  went negative (−1.0, over-lift); `feedback`/`recursion` were +0.0. (P3)
- What predicts the lift's value is **coverage headroom**: it helps when the unaided design
  under-specifies the pattern's components, is neutral when the design is already complete, and can
  *hurt* when an abstract framing is forced onto something the problem handled naturally. (P3, P4)
- The coverage gap is widened by **transport distance** (cousin +0.63 → distant +1.13, P2) and by
  **retrieval burial / low salience** (lift-effect ~2.4× when components are buried among distractors, P4).
- **Forced-frame is not a real risk**: across 13 primes, reasoners decline inapplicable roles *with reasons*
  rather than smuggling the frame. (P2, P3)
- **`lower-to-homolog` ≈ `lift-naive`** so far: both ceiling-bound; "pose the components" is the
  load-bearing move and the extra homolog-search has not separated measurably. (P1–P3) Treated below as a
  *typed mode of lower*, kept but not assumed to add value.
- The §9.2 ablation's lesson — **the pipeline's value was enforced discipline** — is the load-bearing
  caution for this whole project: a free planner may erode that discipline by skipping the helpful step
  (here, the coverage check). The A/B/C ablation is built to isolate exactly this.

Framed/structural typing is still real and still used — it gates **decompose** (§5.4), not lift.

---

## 1. State representation

The planner acts on a single typed **STATE** object that accumulates as verbs are applied. It is the
"typed model + coverage map" the spec calls for, made concrete. Every verb reads some fields and writes
others; post-conditions are predicates over these fields, which is what lets condition C *enforce* them.

```jsonc
State = {
  problem: {
    statement: str,                  // step-1 output: decision, bounds, success/failure criteria
    constraints: [str],
    success_criteria: [str],
    failure_criteria: [str]
  },
  operative_primes: [                // identify → salience-rank → prune
    { slug, type: "structural"|"framed"|"mixed", framedness: 0.0..1.0,
      salience: "load_bearing"|"context", rationale }
  ],
  model: {                           // compose output — the typed relational model
    nodes: [ { id, label, kind: "actor"|"entity"|"resource"|"signal"|"institution"|... } ],
    edges: [ { src, dst, relation, prime_annotation? } ],
    prime_annotations: [ { prime, attached_to: [node_or_edge_ids] } ]
  },
  meta_model: {                      // lift(situation) output — domain-stripped skeleton
    roles: [str], relations: [str], invariants: [str]
    // post-condition: every role/relation has a witness in `model`; contains no domain terms
  },
  candidates: [                      // transport/match output
    { archetype, source_prime_overlap: [slug], retrieval_score, retrieval_route: "meta"|"facet"|"graph" }
  ],
  coverage_map: {                    // *** the lift's verifiable post-condition (the heart) ***
    pattern: <archetype-or-prime slug whose components are being enforced>,
    components: [
      { name,
        status: "instantiated" | "homolog" | "correctly_declined" | "missing",
        witness:        <ref into model/recommendation>,   // REQUIRED iff instantiated|homolog
        decline_reason: str,                                // REQUIRED iff correctly_declined
        gap_note:       str }                               // REQUIRED iff missing
    ]
  },
  gate_log: [                        // evaluate-fit output
    { candidate, anti_signatures: [ { slug, tripped: bool, reason } ], verdict: "keep"|"drop" }
  ],
  recommendation: str | null,        // lower output; emitted only at STOP
  history: [ { step, verb, args, rationale, post_condition: "ok"|"violated"|"n/a" } ],
  cost: { tokens_in, tokens_out, n_operations, n_verifier_passes, wall_clock_s }
}
```

Three design notes:

- **The coverage map is the externalized discipline.** It is the single artifact that makes "did the
  design actually instantiate the pattern's components?" a checkable predicate rather than a vibe. Its
  four statuses are exactly the four fates the experiments surfaced: `instantiated` (the design already
  builds it), `homolog` (a function-discharging mechanism in the target substrate — the P0 separate-cell
  case), `correctly_declined` (the role is genuinely inapplicable *or* the abstract framing would distort
  a part handled naturally — the `symmetry` over-lift back-off), and `missing` (the gap the lift exists to
  catch — the `conservation_laws` dropped-ledger case).
- **`history` is the operation log** that makes the control structure observable and lets us price B/C's
  flexibility. It is also a method-tell, so it is stripped before grading (§5).
- **The STATE is condition-agnostic.** A, B, and C all act on the same object with the same verbs; they
  differ only in *who decides the next verb* and *what must be true to stop*.

---

## 2. Verb library

Each verb: **definition · input/output types · pre-condition · post-condition · checkable core · failure modes**.
Verbs are the same in all three conditions; only sequencing differs.

### match (retrieve)
- **Def.** Anchor the problem to the catalog: given the problem/meta-model, return candidate primes and
  archetypes (neighborhoods, not single hits), using project-01 semantic + type-aware + facet retrieval.
- **In/Out.** `problem.statement | meta_model` → `operative_primes (candidates)`, `candidates (archetypes)`.
- **Pre.** A problem statement exists.
- **Post.** ≥1 candidate returned, each with a retrieval score and route; oblique queries are
  facet-decomposed and retrieved per-facet.
- **Checkable core.** Retrieval is reproducible from the query; each candidate's source-prime overlap is recorded.
- **Failure.** Retrieval failure (apt-but-surface-distant pattern never surfaced); neighborhood crowding.

### salience-rank
- **Def.** Order candidate primes by load-bearing relevance (removing it would change the recommendation).
- **In/Out.** `operative_primes` → same, ranked + tagged `load_bearing|context`.
- **Pre.** ≥1 candidate prime.
- **Post.** Every prime tagged with salience + rationale.
- **Checkable core.** Ablation (drop a prime, re-run lower, confirm the recommendation changes) — expensive, optional.
- **Failure.** Premature de-prioritization of a load-bearing prime.

### prune
- **Def.** Cut the ranked set to the operative subset (dual of salience-rank).
- **In/Out.** ranked `operative_primes` → operative subset.
- **Pre.** A salience ranking exists.
- **Post.** Subset retained (typically 3–9); each cut carries a rationale.
- **Failure.** Premature pruning (cut a prime that carried the dynamics); failure to prune (mush).

### compose
- **Def.** Assemble operative primes into a typed relational model (entities/actors as nodes, relations as
  labeled edges, primes as annotations).
- **In/Out.** `operative_primes + problem` → `model`.
- **Pre.** ≥2 operative primes.
- **Post (referential integrity — checkable).** Every edge references declared nodes; every operative prime
  appears as an annotation.
- **Checkable core.** Graph-validator (mechanical).
- **Failure.** Spurious coupling (invented edges); missing coupling (dropped load-bearing edge).

### lift (abstraction) — **coverage-enforcement, the re-specified verb**
`lift` has two modes that share one checkable artifact.

- **Mode L1 — lift the situation.** `model` → `meta_model` (domain-stripped skeleton). *Post (checkable):*
  witness + neutrality — every meta-model role/relation has a witness in `model`; the meta-model contains no
  domain terms.
- **Mode L2 — lift a pattern's components onto the design (the coverage-enforcing lift).** Given a pattern
  in hand (archetype/prime) and the current design (`model`/draft `recommendation`):
  1. **Name** the pattern's components (archetype `components`, or a prime's structural-signature roles).
  2. **Pose** each as a candidate requirement on the design.
  3. **Coverage-check** each: is it genuinely instantiated in the current design?
  4. **Keep / decline / back-off:** keep components that close a gap; mark `instantiated` the ones the design
     already covers (no double-work); mark `correctly_declined` the ones that are genuinely inapplicable
     **or** whose imposed abstract framing would distort a part the design handles naturally (the over-lift
     guard); mark `missing` the ones that should be there and are not.
  - **In/Out.** `pattern + model` → `coverage_map`.
  - **Pre.** A pattern is in hand and a draft design exists.
  - **Post (THE coverage-map post-condition — checkable).** `coverage_map.components` lists **every**
    component of the pattern; each has a status in {instantiated, homolog, correctly_declined, missing}; each
    `instantiated|homolog` carries a witness ref; each `correctly_declined` carries a reason; each `missing`
    carries a gap-note. *A design may stop only when no component is `missing`.*
  - **Failure.** Over-lift (forcing an abstract frame the problem didn't need — `symmetry`); mis-lift (wrong
    invariant). The over-lift case is **detected by the coverage-check itself** and recorded as
    `correctly_declined` rather than silently degrading the design.
- **Note on the back-off / over-lift detector.** For each posed component, the check asks whether posing it
  as a requirement *adds coverage* or *distorts a sub-structure already handled*. P4 located over-lift in the
  **high-coverage regime** (salient pattern + capable model); the coverage-check is built to skip exactly
  that case (don't lift what's already covered), so the re-specced lift absorbs the one failure mode the
  experiments surfaced.

### lower (instantiation)
- **Def.** Realize a transported pattern/role-set in the target domain (supply the particulars).
- **In/Out.** `pattern/role-set + model` → concrete moves in `recommendation`.
- **Pre.** A pattern in hand + a target.
- **Post.** Every posed component/role maps to a named target mechanism (`instantiated|homolog`) or is
  flagged unsatisfiable (`correctly_declined`) — i.e. lower writes into the same `coverage_map`.
- **Typed mode — lower-to-homolog.** On framed roles, search the substrate for a mechanism that discharges
  the same function even when the literal form is unavailable, and justify its independence. **Kept but
  unproven** (P1–P3: adds quality not feasibility; has not separated from naive lowering — flagged, not assumed).
- **Failure.** Forced lowering; under-specification; naive role-dropping ("independent review" → the unit
  re-checking itself — the original forward-case failure).

### transport (= retrieve + map)
- **Def.** Carry a pattern across a domain gap. Composite: `match` (retrieve candidate) + **map** (align the
  pattern's roles to the situation's entities and report where the correspondence breaks).
- **In/Out.** `meta_model | operative_primes` → an aligned `candidate` + correspondence/break-report.
- **Pre.** A meta-model or operative primes in hand.
- **Post.** Explicit role↔entity correspondence with break-points named.
- **Typed behavior (§5.4).** Structural input → direct recognitional transport (risk: neighborhood crowding).
  Framed input → `lift-roles → map → lower-to-homolog` (risk: surface distance on retrieval; smuggled frame
  on mapping — empirically near-zero per P2/P3).
- **Failure.** Retrieval failure; forced mapping.

### decompose — **the verb framed/structural still gates**
- **Def.** Given a framed prime, strip the frame to expose a structural core (three fates: unifies with an
  existing structural prime / is a new structural prime / is constitutively framed).
- **In/Out.** framed `prime` → `{structural_core, frame}` (or no-op).
- **Pre (TYPED GATE).** `prime.type == framed`. On a structural prime there is no frame to strip → no-op.
  **This is the one place the framed/structural label is load-bearing; it does *not* gate lift.**
- **Post.** A structural core + a stripped frame, or an explicit "constitutively framed" verdict.
- **Checkable core.** Fate-2 verified by **transport**: an extracted core is a real prime only if it projects
  into ≥3 domains where the framed prime looked alien.
- **Failure.** False fate-two (a thin residue mistaken for a projectable pattern).

### evaluate-fit (gate)
- **Def.** Walk a candidate's anti-signatures / trigger conditions / root tensions; drop it if the situation
  trips a disqualifying condition.
- **In/Out.** `candidate + model` → `gate_log` entry + keep/drop.
- **Pre.** Candidate carries anti-signatures (well-typed over **archetypes**; under-typed over bare primes).
- **Post (checkable).** Each anti-signature explicitly evaluated (`tripped?` + reason); candidate dropped if a
  disqualifying condition trips.
- **Checkable core.** §8.4's highest-value target — anti-signatures as explicit conditions over the typed
  model, converting the gate from opinion to rule.
- **Failure.** Hollow gate (nods at the anti-signatures without testing them).

### reconcile
- **Def.** Apply a candidate's logic to the concrete model and to the meta-model separately; resolve the
  (characteristically different) inferences, preferring the reading consistent with both views.
- **In/Out.** inferences from `model` + `meta_model` → a single reconciled reading.
- **Pre.** A candidate has been applied to both views.
- **Post.** Divergences named and resolved.
- **Diagnostic.** If the two views never disagree, the meta-model is decorative — flag it.
- **Failure.** Collapse (one view silently dominates).

**Verb relationships used by the planner.** lift ⊣ lower (adjoint-like pair); decompose ↔ frame (strip vs
add); salience-rank ↔ prune (generate/filter). Cross-checks: decompose is verified by transport; salience by
ablation; lift(L1) by witness-check; compose by referential integrity; the **lift(L2) coverage map** is the
verifiable discipline that condition C enforces at STOP.

---

## 3. Planner mechanism and the A/B/C ablation

**Settled (from the spec): interpreter/planner, not a new fixed procedure; LLM as control policy initially;
clean ablation varying only control structure.**

### The loop (shared skeleton)
The LLM acts as the control policy over the STATE. At each step it observes a STATE summary and emits a
decision `{verb, args, rationale}`. The harness: executes the verb (the same model executes the verb body,
producing the typed output), **validates the verb's post-condition**, appends to `history`, updates `cost`.
A `STOP` decision proposes `recommendation` as final.

### The three conditions (vary only control structure; catalog/verbs/model held fixed)
- **A — Fixed pipeline (baseline).** The nine steps in fixed order, no revisiting:
  specify → match(identify primes) → salience-rank → prune → compose → lift(L1: meta-model) →
  transport/match(archetypes) → reconcile(dual-view) → evaluate-fit → lower(recommendation).
  This is the current pipeline guide verbatim. **Note it has no L2 coverage-enforcing lift step — the step it
  can structurally skip is exactly the A-vs-C crux.**
- **B — Free planner.** Given the verb library + STATE, the model picks the next verb each step; may repeat,
  skip, or stop whenever it judges a recommendation is ready. **No enforced post-conditions** beyond
  "emit a recommendation."
- **C — Flexibility-within-discipline.** Like B, but a `STOP` is **rejected** unless the externalization
  post-conditions hold: (i) compose referential-integrity ok; (ii) lift(L1) witness/neutrality ok;
  (iii) **the lift(L2) coverage check** for the operative pattern is satisfied — coverage map complete, no
  component `missing`; (iv) evaluate-fit gate has been run on the chosen candidate. On any failure the planner
  must continue and apply the missing verb. C is B **plus the verifiable discipline invariant**.

### How C's coverage gate is enforced
This is the design's crux and a genuine fork (see §6, decision 2). The cleaner option — and my recommendation
— is a **separate verifier pass**: an independent model call receives the *design + the operative pattern's
components* (not the planner's self-report) and returns the coverage map. If it marks any component `missing`,
C cannot stop. A separate verifier makes the discipline *enforced* rather than *self-asserted*, which is the
whole point of the §9.2 caution — if C's check were the same context self-grading, B and C would differ only
cosmetically and the experiment would not isolate "enforced discipline."

### Why this isolates the right variable
A, B, C share the scenario, the catalog, the verb set, and the model class. They differ only in *who orders
the verbs* and *what must be true to stop*. So a quality difference is attributable to control structure. The
specific prediction the structure encodes: **A** wastes steps and cannot revisit, but its fixed order also
*omits* the L2 coverage lift; **B** can route around waste but may skip the coverage check when it "feels
done"; **C** keeps B's flexibility but is forbidden to stop on an uncovered design.

### Hypothesis (pre-registered): **C ≥ A > B.**
If **B > A**, probe hard (per the spec): inspect `history` to see whether B won by *incidentally* running the
coverage check anyway (which would argue the discipline, not the rigidity, was A's value and would *support*
C), or by genuinely needing fewer steps on easy scenarios (which would argue A's rigidity is pure overhead).

---

## 4. Experiment design

- **Scenarios.** Full design problems in the fishery mould (problem + decision-required + success/failure
  criteria). **Stratified** on the two axes the experiments say matter for the coverage lift: catalog maturity
  (catalog-strong vs catalog-thin region) and transport distance / **obliqueness** (clean-single-theme vs
  buried-among-many). At least one scenario must be an **oblique/buried** case where A's fixed order should
  skip the L2 lift and C should catch it — otherwise C and A cannot separate. (Count/scope is decision 3.)
- **Reps.** ≥2 per cell (matches prior protocols; single runs are directional — Hubbard).
- **Conditions × scenarios × reps**, each cell a target-blind run producing a normalized recommendation.

### Confound controls (carried from prior runs + new ones the planner introduces)
- **Grade the final recommendation only**, normalized to strip method tells — verb names, "coverage map",
  "archetype", tool names, the whole `history`. Identical normalization for A/B/C so the transcript cannot
  leak the condition.
- **Equal length budget** (e.g. ≤1000 words) across A/B/C, neutralizing length→quality (B/C may otherwise run
  longer).
- **Blinded grading.** Opaque cell ids; private manifest (cid→condition); grader sees only scenario + the
  normalized recommendations + rubric; does not know the condition or the mapping. (Mirrors the protocol1–4 harness.)
- **Rubric overlap (named limitation).** Three of the six dims (coherence, failure-mode anticipation,
  structural/equity fit) are what the catalog enforces. Report those three **separately** as a robustness
  check, as the original kit did, so a C-advantage isn't purely rubric-aligned.
- **Operator contamination.** I (designer) do not grade; grading is independent subagents. Scenarios chosen to
  avoid instance-obliqueness leakage (no scenario telegraphs its operative primes in domain vocabulary).
- **Cost logging.** tokens in/out, `n_operations`, `n_verifier_passes`, wall-clock — to report the *price* of
  B's flexibility and C's discipline alongside quality. A flexibility that buys quality at 3× cost is a
  different verdict than one that's free.
- **Same model class throughout** (the deep designer=solver=grader confound persists; named, not removed).

### Rubric
Keep the six dimensions; **generalize dims 5–6** to scenario-neutral wording (currently fishery-specific):
1 structural coherence · 2 failure-mode anticipation · 3 implementability · 4 mechanism grounding ·
5 stakeholder/equity fit (general: engages actor heterogeneity + the equity floor in the scenario's criteria) ·
6 threshold/time-horizon awareness (general: engages non-linear/collapse risk + the relevant time tension).
Scored 1–10, total /60, one-sentence justification each, plus a forced overall pick (no split-the-difference).

### Decision rule (frozen before runs)
Predict **C ≥ A > B** on total /60. Pre-register the per-scenario expectation that **C's margin over A grows
on the oblique/buried + distant scenarios** (where the coverage lift has the most headroom, per P2/P4) and
**shrinks toward zero on catalog-strong, single-theme scenarios** (where A's fixed order already covers and the
lift is a near-no-op, per P3/P4). If that interaction appears, it is strong evidence the mechanism is the
coverage discipline, not control flexibility per se.

---

## 5. Repeatability

Mirror the protocol1–4 harness exactly so this is reproducible and resumable:
- `scripts/exp_project02.py build|status|pending` — generates `prompts/{cid}.txt` for every cell, writes a
  private `manifest_PRIVATE.json` (cid → {condition, scenario, rep}), tracks completion.
- A `verbs.md` / `verbs.json` shipping the verb specs above as the **shared, versioned** spec all three
  conditions cite (so "verb set held fixed" is literally true and auditable).
- Solver runs as independent subagents (one per cell) with MCP catalog access; the planner loop for B/C is a
  documented prompt + harness; the verifier pass (C) and graders are separate subagents with no shared context.
- All outputs, the manifest, the frozen `PREREGISTRATION.md` (predictions + decision rule), and the final
  `RESULTS.md` versioned under `experiments/project02_engine_2026-05-25/`.

---

## 6. Open decisions for Kurt (these fork the work)

1. **Planner realization** — structured-state loop (model emits explicit `{verb,args,rationale}`, harness keeps
   an explicit STATE, operations logged) **[recommended]** vs. free-form prose (verbs named in prose, STATE
   implicit). The former gives clean op/cost logging and makes C's gate mechanical; the latter is cheaper and
   closer to "raw LLM as policy."
2. **C's coverage-check enforcement** — separate independent verifier pass **[recommended]** vs. in-context LLM
   self-check vs. attempt a mechanical encoding (§8.4). Only a separate pass makes C's discipline *enforced*
   rather than self-asserted — which is what the §9.2 caution requires the experiment to test.
3. **Scenario set size/scope** — ~5 scenarios stratified by catalog maturity × obliqueness, incl. ≥1
   oblique/buried case **[recommended]** vs. a fast 2–3 vs. a larger 6–8.
4. **Grading rigor** — 2 independent graders + 2 reps/cell **[recommended]** (addresses the single-grader
   limitation flagged in every prior protocol) vs. 1 grader/2 reps (cheapest, matches priors) vs. 3 graders on
   a subset for inter-rater agreement.

Everything else follows the settled decisions and the prior harness. On approval I'll finalize `verbs.md`,
build `exp_project02.py`, freeze `PREREGISTRATION.md`, then run A/B/C.
