Confidence Update Log¶
Recordkeeping tool — instantiates Intellectual-Humility Narrative Integration
A dated ledger of how confidence in a claim moved over time, the evidence that moved it, and the decisions each shift fed.
A Confidence Update Log is a running, dated record of one thing: how confident you were in a claim, and how that confidence moved. Each entry captures the belief, the level or band you would put on it today, the evidence that shifted it since the last entry, and the boundary of what that confidence does and does not cover. Its distinguishing move is that it treats confidence as a time series rather than a verdict — the log's value is in the trajectory, not any single reading, because a trajectory reveals whether your certainty tracks the evidence or drifts ahead of it. Kept continuously and internally, a revision here is never a dramatic event but simply the next line.
Example¶
A geopolitical risk analyst is tracking whether a contested border will see armed escalation within six months. In week one she writes the first entry: confidence 30%, basis a troop-movement report of uncertain provenance, scope "covers force posture, not political intent." Three weeks later a second source corroborates the buildup and a summit collapses; she logs 55%, notes the two evidence shifts, and records that this reading fed the desk's decision to move regional exposure to "elevated." A month on, back-channel talks resume; she drops to 40% and writes plainly that she had over-weighted the summit. When the period closes without escalation, the log is not an embarrassment — it is the most useful artifact she owns. Read end to end, it shows exactly where her confidence ran ahead of warrant and which single source she trusted too much, and that becomes the correction she carries into the next assessment.
How it works¶
- One belief per thread, append-only, each entry timestamped so the sequence cannot be quietly rewritten after the fact.
- Each entry carries three fields: the current confidence (a point or a band), the evidence delta since the last entry, and an explicit scope note stating what the claim covers and what it excludes.
- Every shift is cross-referenced to the decision or downstream call it informed, so the record connects a changed belief to the action it changed.
- The whole thread is read back periodically — not to grade the final call, but to inspect the shape of the trajectory: chronic overconfidence, slow drift, a source over-trusted.
Tuning parameters¶
- Update trigger — new-evidence-only versus a fixed cadence. Event-driven keeps entries honest; scheduled review catches slow drift but adds noise.
- Confidence granularity — coarse labels versus numeric probabilities. Numbers enable scoring but invite false precision on claims that cannot bear a number.
- Scope discipline — how strictly each entry must state its boundary. Tight scope prevents overclaiming but slows entry.
- Privacy — personal, team-shared, or open. Sharing adds accountability but can suppress candid low numbers.
- Review cadence — how often the trajectory is read back to inspect calibration.
When it helps, and when it misleads¶
Its strength is that it makes overconfidence visible as a pattern rather than a one-time slip, supports calibration over time, and de-dramatizes revision so a changed mind is routine bookkeeping instead of a confession. Because entries carry probabilities, they can be scored against outcomes with a proper rule such as the Brier score, which is powerful for honest self-assessment.[n1]
Its failure mode is that the log flatters precision — writing "40%" on a claim that cannot support a number — and the scoring that makes it rigorous also tempts you to log only the claims that will score well. The classic misuse is keeping the log to defend a record rather than improve it: retroactive edits, or logging only the easy calls. The discipline that guards against this is append-only entry and a periodic trajectory read-back aimed at the misses, not the hits.
How it implements the components¶
warranted_confidence_boundary— every entry pins the current scope: what the stated confidence covers and what it explicitly does not, so a number is never recorded without its edge.learning_transfer_marker— each shift is cross-linked to the decision it fed, marking the point where an updated belief actually changed practice rather than staying on the page.
It does not stage a public_revision_episode for outside stakeholders or manage the audience_interpretation_frame a change is read through — that is the Public Assumption Revision Memo; the log is an internal ledger of many small updates, not a single published announcement.
Related¶
- Instantiates: Intellectual-Humility Narrative Integration — supplies the longitudinal confidence record the humility narrative can point to.
- Consumes: Epistemic Status Labels — entries reuse its vocabulary to mark each reading's warrant.
- Sibling mechanisms: Public Assumption Revision Memo · Epistemic Status Labels · Humility Norm Onboarding Module · Leader "I Was Wrong" Story · Learning-from-Error Postmortem · Mistake-to-Principle Story Template · Premortem Unknowns Round · Recognition for Belief Revision · Uncertainty Retrospective Prompt
Editorial Notes¶
Form Classification¶
Form family: Record, Log & Register
Rationale: A dated ledger of how confidence in a claim moved over time, the evidence that moved it, and the decisions each shift fed, making its operative form a durable record, ledger, register, or trace whose value depends on preserving actual state or history.
Independent corroboration: The frozen evidence defines Confidence Update Log as 'A dated ledger of how confidence in a claim moved over time, the evidence that moved it, and the decisions each shift fed', so its operative form is Record, Log & Register.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Statistics & Experimental Design
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Probabilistic forecasting practice cohered dated prediction logs and proper scoring against outcomes so calibration is evaluated through time.
Related originating lineages:
- Security Studies & Intelligence Analysis — Analytic tradecraft contributes evidence-linked confidence revisions and bounded scope for unresolved claims.
Review resolution: Both reviewers agree on statistics_experimental_design as primary. Reading the source mechanism confirms that its defining operation belongs to that lineage; the final record retains security_intelligence only where it materially formed the mechanism and keeps present-day application breadth separate from provenance.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Reconciled after independent review; high confidence.
Notes¶
The log and the Public Assumption Revision Memo are easily conflated. The division of labor: the log is where a revision is tracked — continuously, internally, in small increments; the memo is where a single consequential revision is published — once, to an audience. A team that keeps only the memo loses the trajectory; a team that keeps only the log never tells anyone.
[n1] Brier score — a proper scoring rule that measures the accuracy of probabilistic forecasts by the squared distance between the stated probability and the outcome; it rewards calibration and penalizes both over- and under-confidence. ↩