Skip to content

Snapshot Plus Replay

A method — instantiates Event-Log-Centered Modeling

Rebuilds current state fast by starting from a periodic snapshot and replaying only the events since, instead of the whole history.

An event-log model insists that current state is a replay of history — which is elegant until the history is millions of events long and rebuilding state means replaying all of them on every restart. Snapshot Plus Replay resolves that tension without abandoning the log. Periodically it captures a snapshot — the derived state as of a specific point in the log — and stores it alongside a marker for that point. To rebuild, it loads the latest snapshot and replays only the events after the marker, collapsing a full-history replay into a short tail. Its defining property, and what separates it from mechanisms that shrink the log itself, is that the snapshot is a disposable cache of derived state, not a rewrite of history: the full event log stays intact and authoritative, and a snapshot can always be discarded and regenerated by replaying from the beginning.

Example

An event-sourced order-management service holds each order's state as the fold of its lifecycle events — created, paid, partially shipped, refunded, and so on. In its first year this was fine, but the service now carries tens of millions of order events, and every deploy or crash-restart triggers a cold rebuild that replays the entire log into memory — a process that has crept past twenty minutes, turning routine restarts into outages. Snapshot Plus Replay fixes it: a nightly job serializes the in-memory order state and records the log position it corresponds to. On restart the service loads last night's snapshot and replays only today's events — a few thousand instead of tens of millions — and is ready in seconds.

Nothing about the log changed; the twenty-minute full replay is still available and is exactly how a snapshot gets regenerated if one is ever found corrupt or built by buggy code. The snapshot is pure acceleration: a shortcut that the authoritative history can always reconstruct.

How it works

  • Capture state at a known point. Periodically serialize the current derived state together with the exact log marker it reflects, so the pair is an anchored resume point.
  • Resume from the latest snapshot. On rebuild, load the most recent valid snapshot instead of empty state, skipping the history it already encodes.
  • Replay only the tail. Apply just the events after the snapshot's marker to bring state current — the short remainder rather than the whole log.
  • Keep the log whole. Treat snapshots as regenerable derived artifacts; never delete or rewrite the underlying events on their account, so a full replay from origin always remains possible.

Tuning parameters

  • Snapshot interval — how often snapshots are taken. Frequent snapshots shorten the replay tail (fast rebuilds) but cost storage and capture overhead; sparse snapshots are cheap to keep but leave a longer tail to replay.
  • Retention depth — how many past snapshots are kept. Keeping several lets you resume from before a suspected-bad point; keeping only the latest is lean but offers no fallback if the newest snapshot is poisoned.
  • Trigger policy — time-based, event-count-based, or on-demand snapshotting. Event-count triggers bound the tail regardless of traffic bursts; time triggers are simpler but let a busy period balloon the replay.
  • Consistency point — whether a snapshot is taken at a quiescent boundary or from a live, moving state. Quiescent snapshots are cleaner to trust; live ones avoid pausing the system but must pin their marker precisely to stay correct.

When it helps, and when it misleads

Its strength is making an event-log model operationally viable at scale: it preserves the pure "state = replay of history" property while keeping restarts, failovers, and new-consumer bootstraps fast. It is the standard answer to the long-log problem, and it stays honest because the shortcut is always reconstructible from the source.

Its characteristic danger is that a snapshot can silently become a second, wrong source of truth. If a projector bug corrupts state, snapshotting bakes the bug in, and every future rebuild resumes from the poisoned state instead of re-deriving cleanly from the log — the very fault an event-log model is supposed to be able to heal by replay.[1] Schema drift is the other trap: an old snapshot serialized under a superseded state shape can be misread by new code, producing subtle corruption on resume. And a snapshot whose log marker is even slightly wrong replays the wrong tail. The disciplines: treat snapshots as caches that must be validatable against a full replay (pairing well with Projection Rebuild and Diff), version snapshots with the state schema and invalidate on change, and keep enough snapshot history to resume from before a newly-discovered bug rather than being trapped after it.

How it implements the components

Snapshot Plus Replay realizes the archetype's efficient-rebuild slice — making state reconstruction fast — and none of the modelling, correctness, or contested-evidence slices:

  • replay_and_rebuild_path — it is a rebuild path: resume from a snapshot, then replay the tail; the log-to-state derivation is preserved, only accelerated.
  • snapshot_and_compaction_policy — it owns the snapshot half of that policy: when to snapshot, how many to keep, and how to anchor each to a log marker.

It does not guarantee that replaying the tail yields identical results — that determinism is Deterministic Replay Protocol — nor does it shrink the event log itself, which is Log Compaction; the log stays whole and a snapshot is merely a resume point over it.

References

[1] Snapshotting (checkpointing) derived state so that recovery replays only a short suffix of the log is a standard optimization in event-sourced and stream-processing systems. Its accepted caveat is that a snapshot is an optimization, not a record: it must never be trusted over the log, because a snapshot can preserve a defect the full history does not. ↩