Transactional Outbox Projection¶

Write-side consistency protocol — instantiates Access-Optimized Redundant Representation

Writes each source change and an outbox record of it in one local transaction, then relays the outbox to update redundant copies — so a copy is never updated for a write that didn't commit, and never missed for one that did.

A Transactional Outbox Projection solves the write-side hazard at the heart of every redundant copy: how do you update the source and its copies without the two drifting apart when a failure lands between them? The protocol writes the business change and an outbox record describing it inside the same local transaction against the source database. Either both commit or neither does — there is no window where the source changed but the notification was lost, or vice versa. A separate relay then reads committed outbox records and applies them to the redundant projections. Its defining move is anchoring the update-the-copy intent to the source commit atomically, making the source the single authority and the outbox its durable, replayable record of every change the copies must eventually reflect.

Example¶

A payments service records money movements in a ledger and also serves a fast "recent transactions and balance" read model that customers hit constantly. The dangerous naive design is: commit the ledger row, then call the read-model updater. If the process crashes in between, the money moved but the customer's balance projection never learned — the copy silently lies about their balance.

With an outbox, the ledger insert and an outbox row ("balance changed for account A by −$40") commit together in one transaction against the ledger database. A relay polls committed outbox rows and updates the balance projection, marking each row done once applied. If the service crashes after the ledger commit, the outbox row is already durably there, so the relay applies it on recovery — the projection catches up, never skips. Because the ledger transaction is the only thing that can create an outbox row, the ledger stays the sole source of truth, and the projection is provably a downstream copy of committed ledger facts, not an independent tally that can wander.

How it works¶

Commit change and outbox atomically. The source mutation and an outbox record of it are written in one local transaction, so the intent to update every copy is as durable — and as all-or-nothing — as the change itself.
Relay from the committed outbox. A separate process reads committed outbox records and applies them to the redundant projections, decoupling the copies' update from the source transaction's latency.
Deliver at-least-once, apply idempotently. The relay may retry, so each outbox record carries an id or version and projections apply it idempotently — a redelivered change lands once, not twice.

Tuning parameters¶

Relay mechanism — poll the outbox table on an interval, or tail the database change log for outbox inserts. Polling is simple and portable; log-tailing is lower-latency but couples to the database's log format.
Delivery latency vs. batching — how aggressively the relay drains the outbox. Frequent small drains keep copies fresh; larger batches cut overhead at the cost of lag.
Ordering guarantee — global order, per-entity order, or none. Per-entity ordering keeps one account's events in sequence without the cost of serializing everything; stricter ordering is safer but slower.
Outbox retention — how long applied records are kept before pruning. Longer retention aids replay, audit, and rebuilding a projection from history; shorter retention saves space but shrinks the replay window.

When it helps, and when it misleads¶

Its strength is closing the dual-write problem[^dualwrite]: it makes "update the source and its copies" atomic-at-the-source without a distributed transaction, so no committed change is ever lost to the copies and no uncommitted one is ever falsely applied. It is the reliable spine under CQRS read models, search-index updates, and cross-service projections, and its durable log doubles as a replay source for rebuilding a projection from scratch.

It misleads if its guarantees are misread. Delivery is eventual, not instant — the projection lags the source by the relay's cycle, so a reader needing their own write immediately still needs a version gate, not just an outbox. The relay is at-least-once, so a projection that is not idempotent will double-apply a redelivered change and corrupt itself. An un-pruned outbox grows without bound and eventually drags on the source database; a relay that stalls silently lets every copy fall arbitrarily behind while the source looks healthy. And the outbox governs propagation of committed changes — it is not a reconciliation tool, so it cannot repair a copy that has already drifted for other reasons. The discipline is to make every projection idempotent, monitor and bound relay lag, prune the outbox on a policy, and pair it with reconciliation rather than trusting propagation to imply correctness.

How it implements the components¶

Transactional Outbox Projection supplies the write-side authority and synchronization components — how changes leave the source correctly, not the shape or freshness of the copies:

synchronization_rule — it is the rule by which copies track source: capture each change atomically in an outbox at commit, then relay it at-least-once to the projections.
write_authority_rule — all writes go to the source, and only a committed source transaction can emit an outbox record, so projections are strictly downstream and never independently writable.
source_of_truth_reference — the outbox binds every projection update to a committed source fact, keeping the source the single authority the copies derive from.

It does not shape the projections it feeds (redundant_representation_specification) — those are artifacts such as Embedded Aggregate Document or CQRS Read-Model Projection — nor make their freshness observable or gated (version_token, degraded_read_mode), which are Freshness Watermark and Read-Model Version Gate.

Instantiates: Access-Optimized Redundant Representation — the outbox is the write-side protocol that keeps redundant copies faithfully derived from committed source changes.
Sibling mechanisms: Change-Data-Capture Propagation · CQRS Read-Model Projection · Event-Sourced Projection · Synchronization Job · Database-Trigger Synchronization · Read-Model Version Gate

Notes¶

The outbox pattern and change-data-capture solve the same dual-write hazard from opposite ends: the outbox has the application explicitly record intent inside its transaction, while change-data-capture derives the same stream from the database's commit log with no application change. Choosing between them turns on whether you can modify the write path and how much you want copies coupled to the source's log format — a design decision the archetype makes once, not per copy.

References¶

The dual-write problem is the failure mode where an application must update two systems (a database and a copy, queue, or index) with no shared transaction: a crash between the two writes leaves them inconsistent. The transactional outbox is the standard remedy — collapse the two writes into one local transaction plus an asynchronous, replayable relay.