Reflective Error Log¶

Personal error register — instantiates Competence Calibration Feedback

A running log where a person records their own errors and surprises alongside the confidence they held at the time, so patterns of miscalibration surface over the long run.

Reflective Error Log calibrates a person against their own history. Its defining move is to capture real errors and surprises as they happen — each one tagged with the confidence the person held at the moment — and let them accumulate, so that systematic miscalibration emerges from the person's actual track record rather than from a designed drill. Where a Calibration Exercise manufactures prediction moments prospectively, this mechanism harvests them retrospectively from the flow of real work: it is where "I was sure and I was wrong" gets written down and, over time, reveals which situations a person is reliably over- or under-confident in.

Example¶

A security-operations analyst keeps a personal log of alert triage calls that turned out wrong. Each entry is short: the alert, the call she made ("benign, closed it"), her confidence at the time ("high"), and what it actually turned out to be. One entry is a phishing alert she cleared confidently that later proved to be a real intrusion foothold. Individually, each miss is just a bad day. But reviewed at the end of each month, a pattern surfaces that no single incident showed: she is systematically overconfident on alerts involving a particular SaaS integration, clearing them fast because they feel routine. That situation-specific blind spot is invisible in the moment and invisible to a one-off test — it only appears because months of her own confidence-tagged errors are sitting in one place, waiting to be read as a pattern.

How it works¶

Its distinguishing features are that the evidence is self-recorded, real, and confidence-tagged, and that the payoff comes from periodic review. Capturing the confidence held at the time is what makes it a calibration log rather than a mere error list — without it, you learn what you got wrong but not where your certainty betrayed you. And the log is inert until it is mined: the recalibration happens when entries are read across time for systematic, situation-specific patterns. It operationalizes metacognition, turning scattered "huh, I was wrong" moments into a durable, reviewable record.

Tuning parameters¶

Capture threshold — log every judgment versus only surprises and errors. Logging everything captures base rates but is heavy; errors-only is sustainable but can hide how often you were quietly right.
Confidence tagging — whether the prior confidence is recorded. Without it the log tracks errors, not calibration.
Review cadence — how often the log is mined for patterns. Too rare and patterns never surface; too frequent and there's no signal yet to see.
Structure — free narrative versus fixed fields (situation, prediction, outcome, lesson). Structure enables pattern-finding; narrative captures nuance the fields miss.
Privacy — personal versus shared with a coach. Private encourages brutal honesty; shared enables coaching but risks self-censoring.

When it helps, and when it misleads¶

Its strength is unique among the siblings: it is the only one that calibrates on the person's own real errors over time, catching the systematic, situation-specific miscalibration that a single test cannot. It is also cheap, self-directed, and exactly the error-focused feedback loop that deliberate practice depends on.^[1]

It misleads because self-logging is subject to the very blind spots it's meant to expose — you cannot log the errors you never noticed were errors, so the log systematically undercounts unrecognized mistakes. It decays without a review ritual into a write-only diary that changes nothing. And hindsight bias can quietly rewrite the recorded confidence after the fact ("I always knew that was risky"). The discipline is to record confidence before the outcome wherever possible, review on a fixed cadence, and cross-check against external evidence to catch the errors self-observation misses.

How it implements the components¶

Reflective Error Log fills the self-observed, retrospective evidence-and-pattern slice:

performance_evidence_set — the accumulated record of real errors and their outcomes is a domain-relevant evidence set drawn from actual work.
calibration_gap_map — each entry pairs prior confidence with the actual result, and the accumulated pattern is the gap map over time.
recalibration_cadence — the periodic review of the log is the recalibration rhythm itself.

It does not supply an external benchmark (performance_benchmark, Competency Framework) or deliver another person's feedback (calibration_feedback, Peer Review); its evidence is self-observed.

Instantiates: Competence Calibration Feedback — builds a self-owned track record that surfaces systematic miscalibration over time.
Sibling mechanisms: Calibration Exercise · Peer Review · Benchmarked Feedback · Calibration Conversation · Confidence Rating Scale · Competency Framework · Exemplar Comparison · Skills Assessment · Simulation or Case Test · Decision Rights by Competence · Supervised Practice

Notes¶

The log's built-in limit is that you can only record errors you noticed. Its blind spots overlap with the person's own, so unrecognized mistakes never make it into the record. That is why it pairs best with an external evidence source — Peer Review or a Skills Assessment — that can catch what self-observation cannot.

References¶

[1] Deliberate practice, in Anders Ericsson's sense, is focused, effortful practice built around immediate feedback on errors and their correction — not mere repetition. An error log operationalizes the feedback-on-errors part of that loop, giving the practitioner a durable record of exactly where their performance and their confidence diverged. ↩