Skip to content

Checksum and Sample Reconciliation

Correctness audit — instantiates Access-Optimized Redundant Representation

Periodically compares a copy against its source — range checksums plus spot-sampled rows — to detect and quantify divergence, without re-reading every row every time.

Checksum and Sample Reconciliation verifies that a redundant copy still matches its source, and raises a signal — with a magnitude — when it doesn't. Its defining move is that it neither builds nor syncs anything: it audits correspondence. And it does so affordably, by comparing cheap checksums over ranges of the data on both sides and only drilling into individual rows where a range disagrees, so continuous verification doesn't cost a full scan of everything every run. What it produces is evidence — "these rows have drifted, by this much, here" — measured against a correctness objective. It is the mechanism that catches the silent drift every sync pipeline eventually produces, precisely because sync mechanisms cannot reliably detect their own failures.

Example

An operational orders database is copied nightly into an analytics replica that a sync pipeline keeps current. To confirm the replica is trustworthy, a reconciliation job runs each night: for every recent day it computes a checksum — a hash over an ordered projection of that day's rows — on both the source and the replica, and compares. Days whose checksums match are certified without reading a single row. Where a day's checksums differ, the job samples rows in that range and characterizes the difference: missing rows, or a stale field. It reports "≈0.002% of rows across the last 30 days diverge, concentrated in yesterday's late-arriving updates" — a quantified divergence signal against the team's correctness target of ≤0.01%. It does not fix anything; it tells the team, precisely, that and where a repair is owed.

How it works

  • Compare by range, cheaply. Compute a checksum per key-range on both source and copy; matching ranges are certified without row reads.
  • Sample where they differ. Only inside a mismatching range does it pull individual rows, to locate and describe the drift rather than just flag it.
  • Signal against an objective. Emit the divergence and its magnitude relative to a correctness/consistency target, so a breach is defined, not eyeballed.

Tuning parameters

  • Sampling rate — higher coverage catches rarer divergence but costs more per run; the dial trades assurance against expense.
  • Checksum granularity — per-partition (cheap, coarse) vs. finer ranges (localizes drift faster, more overhead).
  • Reconciliation cadence — how often the audit runs; frequent runs shrink the window of undetected drift.
  • Drift threshold — how much divergence trips the alarm — set from the correctness SLO, so "acceptable" is a decision, not an accident.

When it helps, and when it misleads

Its strength is catching the drift that synchronization silently produces — dropped change events, partial failures, derivation bugs — and quantifying it cheaply enough to run continuously. It misleads because sampling gives probabilistic assurance, not proof: a rare, unlucky divergence can slip through a sample, so "reconciled" means "no drift found," not "no drift exists."[1] It also tells you that a range differs, not why. The classic misuse is raising the drift threshold until the check goes green — hiding the divergence instead of repairing it. The discipline is to set the threshold from the correctness objective rather than from what currently passes, and to route every real divergence to an actual repair path.

How it implements the components

Checksum and Sample Reconciliation realizes the detection side of the archetype — it verifies correspondence and reports, without touching the data:

  • divergence_signal — its primary output: a quantified measurement of where and how far a copy has drifted from its source.
  • service_level_objective — it operationalizes a correctness/consistency target (e.g. ≤0.01% divergent rows) and flags when the copy breaches it, turning "trustworthy" into a checked number.

It does not repair the drift it finds (Reconciliation Workflow or a targeted Backfill and Rebuild Job does), run the sync that should have prevented it (Change-Data-Capture Propagation), or define the copy's freshness bound (Freshness Watermark).

Notes

Reconciliation detects; it does not resolve. Pairing it with a repair path — Reconciliation Workflow, or a targeted Backfill and Rebuild Job over the diverged ranges — is what closes the loop. A divergence signal that nothing acts on is just a dashboard that makes everyone feel watched while the copy stays wrong.

References

[1] Merkle-tree / anti-entropy reconciliation — hashing data in a tree of ranges so two replicas can find exactly where they differ by comparing hashes top-down, exchanging only the divergent branches. It is the standard technique (used in Dynamo-style stores) behind comparing large datasets without shipping all of them, and the reason range checksums localize drift so cheaply. ↩