Forecast After-Action Review¶
Learning review — instantiates Anticipatory Forecasting
After the forecasted future has arrived, scores what was predicted against what happened, records the error and its owner, and feeds the lesson back into how the next forecast is made.
Most forecasting loops never close: the future arrives, and no one goes back to check whether the forecast was any good. Forecast After-Action Review is the ritual that closes it. Once the predicted horizon has passed and outcomes are known, it compares forecast to realized, scores the error, attributes it to a named owner and cause, and writes it into a durable record that changes how the next forecast — and its thresholds — are made. Its defining move is that it looks backward to improve forward: it produces no prediction of its own, it produces calibration. Where the forecasting methods look ahead and the warning mechanisms fire in the moment, this one runs after the fact, and its only product is a learned adjustment plus an accountable memory of how the system's forecasts actually perform.
Example¶
A national weather service runs a formal review after every major hurricane season on its track and intensity forecasts. For each storm it lays the official forecast cone against the observed track and scores it — mean track error at 24, 48, and 72 hours, and a Brier score for the probabilistic landfall calls.[n1] The review does not stop at a number. It attributes: the 72-hour track errors clustered on storms that underwent rapid intensification the intensity model consistently under-called, and the responsible forecast desk is named alongside the finding. Those results go into a season-over-season error record, and — the point of the whole exercise — they feed the next cycle: the intensity model's rapid-intensification handling is flagged for revision, and the confidence language in the public product is recalibrated so a "likely" means what the track record says it means. The forecast that was wrong this year is what makes next year's less wrong.
How it works¶
What distinguishes this from any single forecast is that its inputs are realized outcomes and its output is a correction:
- Score forecast against actual. Once outcomes are known, measure error with a proper metric — bias, absolute error, a calibration or Brier score — not an impression of whether it "felt close."
- Attribute cause and owner. Separate error from bad luck versus flawed method or optimism, and name who owns each recurring pattern.
- Accumulate the record. Write each result into a persistent error memory so systematic bias becomes visible across many forecasts, not lost after one.
- Feed the update forward. Convert the finding into a concrete change to the model, the thresholds, or the confidence language used next cycle.
Tuning parameters¶
- Review cadence — per-event versus periodic. Per-event catches specifics while fresh but is heavy; periodic sees systematic patterns but blurs individual causes.
- Scoring metric — which error measure rules. Bias metrics expose systematic over/under-calling; calibration metrics judge whether stated confidence is honest; the choice steers what gets fixed.
- Attribution depth — stopping at "the forecast missed" versus tracing to a specific driver, assumption, or desk. Deeper attribution improves fixes but costs effort and can turn blameful.
- Memory horizon — how many past cycles the error record spans. Longer reveals slow drift and rare-event bias; shorter is nimble but forgets.
- Feedback strength — how firmly findings bind the next forecast, trading disruptive over-correction against lessons that never actually change practice.
When it helps, and when it misleads¶
Its strength is that it is the only mechanism here that makes forecasts get better over time rather than merely be made repeatedly. By accumulating scored error against named owners, it converts scattered misses into visible systematic bias a team can actually correct, and it keeps stated confidence honest.
Its failure modes are about candor and use. Reviewed with hindsight bias, every miss looks predictable and every lesson is really a just-so story, which teaches nothing. Run as a blame exercise, it drives forecasters to hedge and sandbag so they are never caught wrong — corrupting the very forecasts it is meant to improve. And the commonest failure is the review that scores diligently and changes nothing, a scorecard no one acts on. The guarding discipline is to fix the forecast of record before the outcome is known so scoring is honest, separate error from luck with a proper metric, aim attribution at method rather than person, and require every review to name a concrete change to the next cycle.
How it implements the components¶
Forecast After-Action Review fills the archetype's learning slot — it makes no forecast, it improves the ones that do:
forecast_error_memory— its core artifact: the accumulating scored record of forecast-versus-actual that exposes systematic bias.forecast_owner— attributes each recurring error pattern to a named, accountable owner responsible for the fix.update_rule— converts findings into concrete changes to the next cycle's model, thresholds, and confidence language.
It produces no prediction, so it does not implement forecast_target, signal_basis, or uncertainty_range — those are the forecasting methods'. Its nearest twin is Reference-Class Forecast: both touch a record of past outcomes, but the reference class reads a base-rate distribution *forward to make a projection, whereas this review writes the realized-error record backward to recalibrate the next one.*
Related¶
- Instantiates: Anticipatory Forecasting — Forecast After-Action Review closes the loop, turning realized error into a better next forecast.
- Consumes: the outputs of the forecasting mechanisms it scores, such as Demand Forecasting or Trend Projection.
- Sibling mechanisms: Reference-Class Forecast · Demand Forecasting · Trend Projection · Capacity Forecast · Early Warning Forecast · Forecast Trigger Dashboard · Scenario-Informed Preparation · Rolling Forecast Review
Editorial Notes¶
Form Classification¶
Form family: Assessment, Review & Assurance
Rationale: Forecast After-Action Review operates as a bounded evaluation of existing evidence or work that produces a finding or disposition because it after the forecasted future has arrived, scores what was predicted against what happened, records the error and its owner, and feeds the lesson back into how the next forecast is made.
Independent corroboration: The frozen evidence defines Forecast After-Action Review as 'After the forecasted future has arrived, scores what was predicted against what happened, records the error and its owner, and feeds the lesson back into how the next forecast is made', so its operative form is Assessment, Review & Assurance.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Futurism & Strategic Foresight
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Strategic foresight is primary because the review treats a forecast as an explicit anticipatory judgment to be revisited after outcomes. The military after-action-review lineage, statistical scoring, and organizational learning are all formative; combining them into this named review is encyclopedia synthesis.
Related originating lineages:
- Military & Strategic Studies — After-action review supplies the structured learning ritual and ownership of error.
- Organizational & Management Science — Organizational learning institutionalizes accountable review and memory.
- Statistics & Experimental Design — Forecast scoring and calibration provide quantitative comparison against outcomes.
Review resolution: Strategic foresight is primary because the review treats a forecast as an explicit anticipatory judgment to be revisited after outcomes. The military after-action-review lineage, statistical scoring, and organizational learning are all formative; combining them into this named review is encyclopedia synthesis.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Researched adjudication after independent review; high confidence.
Sources consulted:
Notes¶
[n1] The Brier score is a proper scoring rule for probabilistic forecasts — the mean squared difference between forecast probabilities and the binary outcomes that occurred — lower being better. It rewards forecasts that are both confident and correct and penalizes overconfidence, which is why it anchors an after-action review meant to keep stated confidence calibrated rather than merely optimistic. ↩