Confidence Retrospective¶
Retrospective review — instantiates Domain-Specificity of Confidence
Compares past confidence labels against how they actually turned out and repairs the scope map from the pattern of misses.
A Confidence Retrospective is a periodic, backward-looking review: it pulls resolved claims, sets each one's prior confidence label beside its realized outcome, and uses the pattern of hits and misses to edit the scope map — expanding a domain where confidence proved deserved, contracting one where it didn't, splitting a broad domain into finer subdomains, and logging the boundary exceptions it uncovers. Its defining move is closing the loop: turning accumulated outcome history into revisions of where confidence is licensed. It is retrospective, and it works from realized outcomes that a forward-looking transfer check, by definition, never has.
Example¶
A venture partnership runs a semiannual retrospective. It lays each past investment's conviction score, recorded at the time, against realized outcome, bucketed by sector. A pattern emerges: high-conviction "consumer" bets underperformed while lower-conviction "infrastructure" bets over-delivered. The retrospective doesn't just note the miss — it acts on the map. It splits the sprawling "consumer" domain into "consumer-social" (where the partnership was systematically overconfident) and "consumer-fintech" (where it was well calibrated), downgrades the former, and registers two boundary exceptions where an out-of-scope bet unexpectedly worked and is worth studying rather than dismissing. The partners guard the session against reading the past as more obvious than it was.[n1]
How it works¶
- Gather resolved claims with their original labels. The unit is the recorded prior confidence, not what anyone now remembers believing.
- Score label against outcome, by domain. Look for systematic over- or under-confidence within each subdomain, not just individual misses.
- Emit edits to the scope map. The output is concrete revisions — expand, contract, or split domains — fed back to whoever holds the standing map.
- Register boundary exceptions. Cases that crossed a scope boundary and then succeeded or failed are logged as a standing record, so anomalies inform the next revision instead of vanishing.
Tuning parameters¶
- Cadence — how often it runs; frequent reviews are responsive but noisy, rare ones let the map go stale.
- Look-back window — how much history each session weighs; a long window is stable, a short one adapts faster to changed competence.
- Edit aggressiveness — how much evidence is required before a domain is split or a band moved; aggressive editing adapts quickly but over-fits to recent luck.
- Exception threshold — what counts as a boundary exception worth registering; a low threshold captures more anomalies and more noise.
- Blind scoring — whether outcomes are graded without seeing the original label; blind grading is the main defense against rewriting the past.
When it helps, and when it misleads¶
Its strength is that it is the only mechanism that keeps the whole system from going stale: without it, scope boundaries freeze after learning, so the map slowly diverges from reality. It is the archetype's update engine.
Its signature failure is hindsight bias — the retrospective rewrites what "we obviously knew all along," laundering luck into skill and blame into narrative, which corrupts every map edit that follows.[n1] Its other failure is over-fitting: yanking a band on the strength of a handful of recent outcomes. The classic misuse is turning the session into a blame tribunal, which makes people defend past confidence rather than surface real misses. The guarding discipline is to score against the recorded prior label rather than memory, require a minimum sample before moving a band, and hold learning strictly separate from accountability.
How it implements the components¶
The Confidence Retrospective fills the update-and-repair side of the archetype:
feedback_update_loop— its core: it converts outcome history into revisions of confidence, closing the loop the rest of the pattern leaves open.boundary_exception_register— it logs the cases that crossed a scope boundary and what happened, as a standing register that feeds the next revision.source_domain_map— its output is a repaired scope map: source domains expanded, contracted, or split finer in light of measured outcomes.
The retrospective reasons from realized outcomes and never evaluates a not-yet-taken leap. Naming a target and grading its transfer assumptions before the fact — target_domain_definition and transfer_validity_check — belongs to the Transfer Assumption Review, its forward-looking twin. The review asks "will confidence transfer?"; the retrospective asks "did it?"
Related¶
- Instantiates: Domain-Specificity of Confidence — the retrospective is the feedback step that keeps every boundary current.
- Consumes: Track Record by Domain Scorecard supplies the resolved outcome data; Claim Confidence Labeling supplies the prior labels being scored.
- Sibling mechanisms: Expertise Scope Matrix · Claim Confidence Labeling · Out-of-Domain Prompt · Track Record by Domain Scorecard · Transfer Assumption Review · Referral or Collaboration Protocol
Editorial Notes¶
Form Classification¶
Form family: Assessment, Review & Assurance
Rationale: Compares past confidence labels against how they actually turned out and repairs the scope map from the pattern of misses, making its operative form a bounded evaluation of existing evidence or work that produces a finding or disposition.
Independent corroboration: The frozen evidence defines Confidence Retrospective as 'Compares past confidence labels against how they actually turned out and repairs the scope map from the pattern of misses', so its operative form is Assessment, Review & Assurance.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Statistics & Experimental Design
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Forecast-evaluation practice cohered retrospective comparison of recorded confidence with outcomes to recalibrate where judgments are reliable.
Related originating lineages:
- Psychology — Hindsight-bias and metacognition research supply the need to grade recorded priors rather than reconstructed memory.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Independent reviewer agreement; high confidence.
Notes¶
The retrospective's edits flow into the Expertise Scope Matrix: though the matrix reads as the upstream, standing artifact, it is downstream of this review, which is what keeps its cells from silently decaying. Scorecard and retrospective are a division of labor, not a duplication — the scorecard is the continuous ledger that measures accuracy; the retrospective is the periodic event that acts on the ledger to repair the map.
[n1] Hindsight bias — the tendency, after an outcome is known, to perceive it as having been predictable ("I knew it all along"). It threatens any retrospective by inflating past certainty and turning luck into apparent skill; grading against recorded prior labels rather than recollection is the standard countermeasure. ↩a ↩b