Decision Trace Review¶

Review — instantiates Tacit Knowledge Elicitation

Reconstructs an actual decision sequence from the records and artifacts it left behind, then walks the practitioner through the timeline to recover what they knew, and chose, at each step.

Memory of a decision is a reconstruction; the trail a decision leaves — logs, messages, timestamps, drafts, tool states — is evidence. Decision Trace Review starts from that evidence, rebuilding the actual sequence of what happened when, and only then brings the practitioner in to narrate it. Its defining move is to be artifact-anchored: the timeline comes first, assembled from records, so the practitioner is reacting to what they actually did and knew at each moment rather than to a smoothed recollection. This pins down the information available at each step — including how partial and ambiguous it was in real time — and exposes the exact points where the practitioner departed from the expected path. It is the mechanism for the question "walk me through what you actually did, minute by minute, and tell me what you were seeing."

Example¶

A site-reliability team reviews a production outage — not with a "what did you do?" retrospective, but by first reconstructing the trace from artifacts: the alert timestamps, the chat log, the commands run, the dashboards as they looked at each minute, the deploy history. Laid out as a timeline, the record shows the on-call engineer did not follow the standard runbook: eleven minutes in, instead of rolling back as procedure dictates, she started tailing a specific log stream.

Walked through her own trace, she recovers the tacit reasoning: at minute eleven the error rate was climbing but the shape of the errors (all from one shard, all timeouts, not 500s) told her a rollback would mask a data-layer problem that would resurface worse. Nothing in the runbook encodes that discrimination — it lived in her decision to deviate, which the trace makes undeniable and locatable. The review recovers when and from what information she broke the rule, and why. What the artifact trail can't supply on its own is that reasoning; the practitioner must narrate it — but anchored to the record, her account can't quietly drift into the tidier story hindsight would prefer.

How it works¶

Reconstruct from artifacts first. Assemble the actual sequence from logs, messages, timestamps, versions, and tool states before asking the practitioner anything — the record sets the timeline, not the memory.
Recover the information-at-each-step. For each decision point, pin down what was actually knowable then, resisting the pull to import what everyone knows now.
Locate the departures. Compare the trace to the expected or prescribed path; the points of divergence are where tacit judgment acted and are the review's highest-value targets.
Narrate against the record. Have the practitioner explain each step while looking at the trace, so their account stays tied to the evidence rather than floating free.

Tuning parameters¶

Trace completeness — how fully the sequence can be reconstructed from artifacts. Rich instrumentation yields a tight trace; sparse records leave gaps the practitioner must fill from memory, reintroducing reconstruction bias.
Reconstruction granularity — minute-by-minute or milestone-by-milestone. Finer traces catch subtle timing judgments but cost more to assemble and can drown the signal in noise.
Practitioner involvement — trace-only forensics, or trace-plus-narration. Adding the practitioner recovers the reasoning; omitting them keeps the account purely evidential but loses the tacit layer.
Divergence focus — review the whole trace evenly or zoom on the departures from expected practice. Zooming is efficient for eliciting tacit judgment but can miss the ordinary steps that give the departures meaning.

When it helps, and when it misleads¶

Its strength is objectivity of sequence: because the timeline is pinned to artifacts, it resists the smoothing that pure recall imposes, and it makes the departures from prescribed practice both undeniable and precisely located — which is exactly where tacit expertise shows itself.

Its defining hazard is hindsight bias: reviewing a decision when the outcome is already known makes the right path look obvious in retrospect and the information available at the time seem clearer than it was, so both reviewer and practitioner over-credit or over-blame the choice.^[1] The trace also only captures what left a trail — the glance out a window, the gut feeling, the hallway conversation may be the real driver and appear nowhere in the logs. And a review run to assign blame rather than elicit knowledge poisons the practitioner's candor and turns the trace into a courtroom exhibit. The discipline that keeps it honest is to reconstruct the information-available-then before revealing the outcome's meaning, hold a blameless framing, and treat the artifact trace as the skeleton the practitioner's narration must flesh out — not as the whole story.

How it implements the components¶

Decision Trace Review fills the components that concern the reconstructed real sequence:

decision_trace — its core output: the artifact-anchored, time-ordered reconstruction of what was decided when and on what information.
exception_probe — comparing the trace to the expected path surfaces exactly where and why the practitioner departed from the rule.

It works from an actual recorded sequence, not from memory of one episode (decision_rationale_probe on recall — Cognitive Interview) and not from live observation (expert_practice_observation — Shadowing Session); it does not curate the result into a reusable corpus (critical_incident_case — Case Library Capture).

Instantiates: Tacit Knowledge Elicitation — Decision Trace Review recovers tacit judgment from the evidence a real decision leaves behind.
Sibling mechanisms: Critical Incident Technique · Cognitive Interview · Expert Debrief · Cognitive Task Analysis · Case Library Capture · Think-Aloud Protocol · Shadowing Session · Apprenticeship Observation · Simulation Replay · Judgment Aid · Task Analysis

Notes¶

Decision Trace Review depends on the decision having left a usable trail. In highly instrumented, artifact-rich work (software operations, trading, clinical systems) it is powerful; in work that leaves little record (a hallway triage, a craftsperson's adjustments) the trace is too thin to anchor, and a memory-based method like Cognitive Interview or a live method like Shadowing Session is the better fit.

References¶

[1] The tendency to see a past outcome as having been more predictable than it actually was — hindsight bias (Fischhoff) — is the standard reason retrospective decision review distorts the reconstructed rationale. Establishing what was knowable at each step before letting the outcome color the account is the recognized corrective, which is why the method reconstructs information-available-then first. ↩