Prediction and Surprise Log¶
Running calibration log — instantiates Other-Agent State Model Calibration
A running record of what the other agent was predicted to do, what they actually did, and how the model changed — making calibration visible across repeated interactions.
Prediction and Surprise Log is the only mechanism here that writes the prediction before the outcome and then keeps score. Where an empathy map is a snapshot and an after-action review is a periodic meeting, the log is a continuous, forward-dated record whose entire value is temporal: by pairing each confidence-tagged prediction about the other agent with what actually happened, it turns surprise from a feeling into a measurable signal. Its defining move is to make the model's accuracy auditable over time — to reveal whether your read of an agent is genuinely improving or merely growing more confident.
Example¶
A union bargaining team keeps a prediction-and-surprise log across sessions. Before one session they record a confidence-tagged prediction: "Management rejects the wage ask outright (≈70%), because we read their binding constraint as the board, not the budget." Management instead opens with a phased-increase counter — a surprise against a high-confidence call. Logged. Over roughly five rounds a pattern emerges from the entries: the team consistently over-predicts hardline refusals, because their model keeps assuming a political constraint that never materializes. That recorded surprise pattern is what finally revises the hidden-state hypothesis — management's real constraint is timing, not the board — which reshapes the next offer. No single session would have shown this; the calibration only became visible once the misses were written down and compared.
How it works¶
- Records a prediction ahead of time, with an explicit confidence, tied to the specific hypothesis about the agent's state it rests on.
- After the interaction, records what actually happened and reads off the surprise — a high-confidence prediction violated is the informative case; a hedge that comes true teaches nothing.
- Accumulates across interactions so patterns of miscalibration surface — which states you systematically over- or under-read.
- Feeds each surprise back as an update to the named hypothesis that failed, not a vague resolution to "be less sure."
- Timestamps predictions before outcomes so they cannot be quietly rewritten into "I knew it all along."
Tuning parameters¶
- Confidence granularity — coarse (low / medium / high) versus numeric probabilities. Numeric enables real calibration scoring but invites false precision on a soft read.
- Logging threshold — log every interaction versus only the consequential or surprising ones. Logging all catches drift but is heavy; logging only surprises biases the record toward misses.
- Surprise sensitivity — how large a prediction error counts as a "surprise" worth an update. Too sensitive and the model thrashes on noise; too dull and real drift hides.
- Attribution grain — whether a miss updates one specific hypothesis or the whole model. Fine attribution learns faster but can overfit to a single odd interaction.
- Review pairing — whether the log is read solo or fed into an after-action review; pairing turns private entries into shared rule changes.
When it helps, and when it misleads¶
Its strength is that it is the archetype's defense against hindsight bias: because the prediction is written and dated before the outcome, the log stops you from remembering "I knew they'd react that way" after the fact, and it converts the vague sense that your read is good into an auditable track record.[n1] In repeated interactions it turns every encounter into a calibration data point.
Its failure modes follow from the same design. The log is only as honest as its confidence labels — vague, unfalsifiable predictions ("they might push back") can never be surprised and so never teach — and a log kept only when things go wrong overstates your inaccuracy. It can also nudge you toward treating a person as a prediction target rather than an agent to understand, and it is easily run backwards, with predictions padded in hedges so nothing ever counts as a miss. The discipline is to predict specifically and falsifiably, log the boring confirmations too, and use surprise to revise a named hypothesis rather than to congratulate or flog yourself.
How it implements the components¶
Prediction and Surprise Log realizes the calibration-over-time side of the archetype — it measures whether the model is accurate, without gathering the evidence or acting on it:
uncertainty_and_confidence_annotation— every prediction carries an explicit confidence; the log is built around confidence because surprise is defined relative to it.hidden_state_hypothesis_set— each prediction is a bet on a specific hypothesis about the agent's state, so a miss points at exactly which hypothesis to revise.feedback_update_loop— recorded surprise is the update signal; the log is the instrument that captures it, interaction by interaction.
It does not gather the underlying evidence — that is Perspective-Taking Interview's job — and it does not, by itself, rewrite interaction rules or repair harm; acting on the accumulated signal belongs to Interaction After-Action Review, which reads the log.
Related¶
- Instantiates: Other-Agent State Model Calibration — the log is the calibration record that keeps the model's accuracy visible over time.
- Consumes: Belief-Desire-Knowledge Map (or whatever standing model is in use) supplies the hypotheses whose predictions the log tests.
- Sibling mechanisms: Interaction After-Action Review · Empathy Map with Evidence Marks · Perspective-Taking Interview · Belief-Desire-Knowledge Map · Active Listening Loop · Counterparty Model Red Team · False-Belief Check · Role-Reversal Simulation · Consent and Privacy Boundary Checklist · Stakeholder Hidden-Constraint Board
Editorial Notes¶
Form Classification¶
Form family: Record, Log & Register
Rationale: Prediction and Surprise Log operates as a persistent ledger, log, register, or case record that preserves history and traceability because it a running record of what the other agent was predicted to do, what they actually did, and how the model changed — making calibration visible across repeated interactions.
Independent corroboration: The frozen evidence defines Prediction and Surprise Log as 'A running record of what the other agent was predicted to do, what they actually did, and how the model changed — making calibration visible across repeated interactions', so its operative form is Record, Log & Register.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Psychology
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Recording expected and observed behavior to update a model belongs to judgment, learning, and social-cognition psychology.
Related originating lineages:
- Cognitive Science — Computational theory-of-mind and prediction-error models materially shape explicit model updating.
- Organizational & Management Science — Organizational management contributes the persistent register and review cadence across interactions.
Review resolution: Both blind reviewers agree that psychology is the primary origin. Reconciliation resolves reported ambiguity, alternate origin disagreement. Formative alternate lineages are retained as organizational_management, cognitive_science; later breadth of use is recorded separately as domain_reach=multi_domain, while origin_mode=cross_disciplinary_synthesis describes the relationship among origin lineages.
Attribution caveat: The running register is an encyclopedia-specific artifact built from established prediction and calibration ideas.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Reconciled after independent review; medium confidence.
Notes¶
The log's power and its risk are the same feature: it reduces a person to a sequence of predictions. That is exactly right for calibrating your own accuracy, but it must not become the model itself — a well-kept log can be perfectly calibrated about what an agent does while missing why, which is the part that actually guides humane action. Keep it paired with a state model, never a substitute for one.
[n1] Hindsight bias — after an outcome is known, people misremember having expected it all along, which destroys the evidence needed to tell a good read from a lucky one. Writing predictions down and dating them before the outcome is the standard defense, and it is precisely what this log does. ↩