Idempotency and Deduplication Ledger¶
Artifact — instantiates Bidirectional Consistency Mapping
Records applied change identities and outcomes so retry does not compound effects.
An Idempotency and Deduplication Ledger is the durable table the receiving side consults at apply time to answer a single question: have I already applied this change? Its defining idea is that safety against retry lives in a record of what has been done, not in the marking on the wire — the ledger remembers each change's identity and its outcome so that a second delivery of the same change is recognized and skipped, applied once and only once regardless of how many times it arrives. It is a passive artifact, a lookup and an append, consulted by many mechanisms; it does not capture changes, does not transform them, and does not decide who is authoritative. It is memory, and its whole value is that the memory outlives the delivery.
Example¶
A payments API charges a customer's card. The client, unsure whether its first request succeeded before the connection dropped, retries the charge — twice. Each request carries the same idempotency key, pay_2f9c1. On the first arrival the service records pay_2f9c1 → charged \$40.00, receipt r-771 in the ledger before returning. When the two retries arrive, the service finds pay_2f9c1 already present, and instead of charging the card again it returns the stored outcome — the same receipt r-771. The customer is charged forty dollars once, not one hundred and twenty, and every retry gets an identical, truthful answer.
The outcome that matters is what the ledger stores beyond a bare "seen" flag: the result of the first application. That is what lets a duplicate return the original receipt rather than merely being dropped, which in turn is what makes at-least-once delivery safe to build on. Without the stored outcome, a retry could be silently ignored and the caller left not knowing what happened.
How it works¶
- Check before apply. The receiver looks up the incoming change's identity in the ledger before doing anything; a hit short-circuits to the stored outcome.
- Record identity plus outcome. On first application it writes the change identity, the mapping version under which it was applied, and the result, atomically with (or before) the effect.
- Key on causal identity, not value. Two independent edits can produce equal values; the ledger dedups on change identity, so it never mistakes a real second edit for a duplicate.
- Bounded, auditable compaction. Old entries are compacted on a retention rule long enough to outlast the maximum realistic redelivery and offline window, never expired on a whim.
Tuning parameters¶
- Identity key — client-supplied idempotency key, content hash, or composite of source plus sequence. Client keys are precise but require discipline; hashes are automatic but collide on legitimately identical intents.
- Retention horizon — how long an applied identity is remembered. Must exceed the longest realistic retry and offline delay; longer is safer but costs storage.
- Outcome richness — store just "applied," or the full result to replay to duplicates. Rich outcomes make retries fully answerable but enlarge the ledger.
- Write ordering — record-then-apply versus apply-then-record, and how a crash between them is reconciled; the choice sets whether the failure mode is a rare double-apply or a rare lost outcome.
When it helps, and when it misleads¶
Its strength is making retry boring. Because at-least-once delivery is the normal, correct behavior of any durable pipeline, something must absorb the duplicates it produces — and a ledger keyed on change identity does exactly that, turning "the message came twice" into a no-op with a truthful reply.[n1]
Its failure mode is a retention horizon set too short: if an identity is compacted before the last straggling redelivery arrives, that redelivery looks new and the effect compounds — the exact bug the ledger exists to prevent, now with a false sense of safety. A second misuse is deduping on value equality, which drops a genuine second edit that happened to produce the same value. The guarding discipline is to size retention against the real offline and retry window and to key strictly on causal identity.
How it implements the components¶
echo_idempotence_and_fixed_point_guard— it is the idempotence half of the guard: the durable record that makes reapplying a change a no-op, so retries and replays cannot compound effects.causal_change_identity_and_origin— it stores each change's stable identity (and mapping version) as its key, distinguishing a redelivery from an independent edit that merely looks the same.
It does not implement the echo-suppression side of that guard on the wire — recognizing a propagated change as one's own comes from its complement, the Synchronization Origin Token, which the ledger's key can incorporate but does not produce.
Related¶
- Instantiates: Bidirectional Consistency Mapping — the durable memory that makes duplicate delivery harmless.
- Consumes: Synchronization Origin Token — the token's idempotency identity is a natural ledger key.
- Sibling mechanisms: Dual-Write Outbox and Inbox Pattern · Synchronization Origin Token · Tombstone and Revocation Propagation · Bidirectional Change-Data-Capture Adapter
Editorial Notes¶
Form Classification¶
Form family: Record, Log & Register
Rationale: Idempotency and Deduplication Ledger operates as a durable record, ledger, register, or trace whose value depends on preserving actual state or history because it records applied change identities and outcomes so retry does not compound effects
Independent corroboration: The frozen evidence defines Idempotency and Deduplication Ledger as 'Records applied change identities and outcomes so retry does not compound effects', so its operative form is Record, Log & Register.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Computer Science & Software Engineering
Origin pattern: Single lineage
Present-day reach: Specialized
Rationale: Server-side storage of idempotency keys and prior results is a distributed-systems pattern for effectively-once effects under redelivery.
Review resolution: Both reviewers independently assign computer_science as the primary originating domain, so that shared primary is retained. Alternate domains are the union of reviewer-identified formative or independently originating lineages; later application settings alone are excluded. The evidence describes one principal historical lineage. Its defining controls and vocabulary remain bounded to a particular professional or technical practice. The encyclopedia entry generalizes the established mechanism without creating a new composite lineage.
Review outcome: Reconciled after independent review; high confidence.
Notes¶
[n1] An idempotency key is a caller-supplied identifier attached to a request so the server can recognize retries of the same intended operation and return the original result instead of performing it again. The ledger is the server-side store of those keys and their outcomes; it is what makes an operation idempotent under redelivery. ↩