Prediction Journal¶
Logging artifact — instantiates Heuristic Calibration and Confidence Judgment
An append-only record that captures each judgment, its stated confidence, and its resolution date at claim time, so a real track record can accrue instead of a remembered one.
A Prediction Journal is the humble artifact that makes every downstream calibration possible: an append-only log where each judgment is written down before its outcome is known, together with the confidence attached to it and the date it will resolve. Its defining property is the timing — it captures the claim prospectively, at the moment of judgment, locking in what was actually believed and how strongly. Without it, calibration runs on memory, and memory is a liar: successes are vivid and misses evaporate, so a heuristic feels far more reliable than the ledger would show. The journal exists to defeat that selective recall by creating an immutable record no one can retroactively edit. It does not analyze anything; it is the raw material that a bucket review bins, a reliability diagram plots, and a recalibration loop scores. It only guarantees the data will be honest and complete.
Example¶
A startup CEO makes constant fast judgment calls — "this hire will work out," "this feature ships by Q3," "this competitor won't enter our segment" — each with an implicit confidence he never records. He starts a prediction journal. For every consequential call, he writes one line: the claim, a confidence (say 80%), and a resolution date. Crucially he writes it at decision time, so there is no wiggle room later. Six months on, the journal has forty resolved entries. Reading them back is deflating and useful: his "90%+" calls resolved true only about two-thirds of the time, and his optimism was worst on schedule predictions. He remembered being a sharp judge; the ledger says he is overconfident on timelines. That record — not his recollection — is what any real calibration must be built on. The journal did not tell him how to fix it; it just refused to let him forget what he actually predicted.
How it works¶
The mechanism is discipline plus a schema, not analysis. Each entry captures four things at claim time: the judgment stated precisely enough to be unambiguously resolvable ("ships by Sept 30," not "ships soon"), the confidence in a consistent format, the resolution date, and enough context to reconstruct the reasoning later. The entry is append-only and time-stamped — the anti-tampering property is the point. Then the loop half kicks in: at each resolution date the outcome is recorded against the original entry, so the journal steadily fills with matched claim-confidence-outcome triples. That accumulating, un-cherry-picked record is the journal's entire output. It performs no bucketing, no plotting, no re-fitting; its contribution is to ensure that when those analyses run, they run on complete prospective data rather than a flattering highlight reel.[n1]
Tuning parameters¶
- Capture threshold — which judgments get logged. Log everything and the journal is comprehensive but onerous, and trivial calls drown the signal; log only "important" calls and you reintroduce the selective memory the journal exists to defeat.
- Confidence format — numeric probabilities, a fixed vocabulary of bands, or ranges. Numbers are precise and analyzable but invite false precision; coarse bands are easier to sustain but blunt the eventual calibration read.
- Resolvability standard — how strictly each claim must be phrased to have an unambiguous outcome. A strict standard makes scoring clean but slows logging; a loose one is fast but leaves entries that can never be fairly graded.
- Resolution discipline — how reliably outcomes are actually recorded when they land. Weak follow-through leaves entries open and biases the resolved subset toward memorable cases.
When it helps, and when it misleads¶
Its strength is foundational: it converts fuzzy self-belief into a hard, complete, tamper-resistant record, and in doing so neutralizes the hindsight bias and selective memory that make un-journaled judges feel better calibrated than they are. Every other mechanism in this archetype is more trustworthy when fed by a journal.
Its failure mode is silent incompleteness: if only some judgments get logged, or only some resolutions get recorded, the journal inherits exactly the selection bias it was meant to kill — and now it looks rigorous, lending false authority to a skewed sample. The classic misuse is vague claims that cannot be fairly resolved ("things will improve"), which fill the journal with un-gradable entries and let the author quietly score ambiguous outcomes as wins. The guarding discipline is a strict resolvability standard at entry time, a consistent capture rule so the logged set isn't cherry-picked, and religious recording of outcomes — including the misses that memory would rather drop. A journal is only as honest as its most-skipped entry.
How it implements the components¶
heuristic_track_record_evidence— the journal is the track record: the accumulating, prospective ledger of claims, confidence, and resolved outcomes that all calibration analysis reasons over.confidence_claim_format— it fixes and enforces the format in which confidence is recorded, so entries are comparable across time and analyzable later.feedback_collection_loop— recording each outcome against its original entry at the resolution date is the intake half of the loop, steadily pairing predictions with what happened.
It does not bin, plot, or re-fit anything from that record. Grouping calls by stated-confidence bin and auditing hit rates is bias_and_miscalibration_probe work owned by Confidence Bucket Review; scoring resolved outcomes to re-fit controls under a calibration_owner is Post-Outcome Recalibration Review, which consumes this journal.
Related¶
- Instantiates: Heuristic Calibration and Confidence Judgment — supplies the prospective, tamper-resistant record every other calibration mechanism depends on.
- Sibling mechanisms: Calibration Adjustment Rule · Challenge Case Set · Confidence Bucket Review · Ecological Validity Screen · Expert Disagreement Calibration · Low-Confidence Escalation Trigger · Post-Outcome Recalibration Review · Reference Class Comparison · Reliability Diagram or Calibration Curve
Editorial Notes¶
Form Classification¶
Form family: Record, Log & Register
Rationale: Prediction Journal operates as a persistent ledger, log, register, or case record that preserves history and traceability because it an append-only record that captures each judgment, its stated confidence, and its resolution date at claim time, so a real track record can accrue instead of a remembered one.
Independent corroboration: The frozen evidence defines Prediction Journal as 'An append-only record that captures each judgment, its stated confidence, and its resolution date at claim time, so a real track record can accrue instead of a remembered one', so its operative form is Record, Log & Register.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Psychology
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Capturing predictions and confidence before resolution is rooted in judgment-calibration psychology.
Related originating lineages:
- Futurism & Strategic Foresight — Strategic foresight developed forecasting tournaments and disciplined prediction practice.
- Statistics & Experimental Design — Statistics supplies scoring and calibration of stated probabilities.
Review outcome: Independent reviewer agreement; high confidence.
Notes¶
The journal's value is almost entirely in the prospective capture; a "journal" reconstructed after outcomes are known is not a journal but a rationalization, because the very confidence it records has already been contaminated by knowing how things turned out. If you did not write it down before you knew, it cannot calibrate you.
[n1] Hindsight bias — studied by Baruch Fischhoff — is the tendency, once an outcome is known, to believe one "knew it all along," which systematically inflates recalled confidence in correct calls and erases the surprise of wrong ones. A prospective journal is the standard defense: it fixes the claim before the outcome can rewrite the memory of it. ↩