Durable Queue with Acknowledgement¶
Messaging infrastructure tool — instantiates Message-Mediated State Coordination
Persists each message and keeps it until the consumer acknowledges success, redelivering on crash or timeout — so messages survive failure, at the cost of possible duplicates.
A Durable Queue with Acknowledgement stores every message on stable storage the moment it is accepted and does not consider it delivered until the consumer sends an explicit acknowledgement that processing finished. If the consumer crashes mid-work, times out, or negatively acknowledges, the message is redelivered rather than lost. The idea that makes it this mechanism: delivery is provisional until confirmed by a receipt. This buys at-least-once delivery across process and broker failures — the message gets through even when things crash — and its defining trade-off follows directly: a message can be processed more than once (crash after the work, before the ack), so consumers must be idempotent. It answers "will this message survive a crash?", not "what does the message mean?"
Example¶
A card-payment platform must never silently lose a SettlePayment message — money depends on it. The message lands in a durable queue that immediately writes it to disk and replicates it. A worker picks it up (the broker marks it in-flight, not gone), calls the card network, and only after the settlement confirms does it acknowledge, at which point the broker deletes the message. Now suppose the worker crashes right after settling but before the ack lands: the broker's acknowledgement timeout expires and it redelivers the same message to another worker. Without care the payment would settle twice — so the worker keys the operation on the payment's idempotency key and the second attempt is a no-op (that deduplication is Retry with Idempotency Key's contribution, consumed here). Real mechanisms of this shape include RabbitMQ's publisher confirms with consumer ack/nack and SQS's visibility-timeout-then-redeliver. The outcome: a payment is never lost to a crash, and — paired with idempotent consumers — never applied twice either.
How it works¶
- Persist on accept. The message is written to durable storage, often replicated, before the sender is told it was accepted, so a broker restart cannot lose it.
- Lease, don't hand off. A consumer taking a message holds it in-flight — invisible to others but not deleted; it returns to the queue if not acknowledged in time.
- Delete only on ack. The message leaves the queue only when the consumer confirms success; a nack or a timeout triggers redelivery.
- Bound the redelivery. After a set number of failed attempts the message is routed out of the main flow rather than redelivered forever.
Tuning parameters¶
- Durability level — in-memory versus fsync-to-disk versus replicated quorum. Stronger durability survives worse failures but adds write latency to every message.
- Acknowledgement timeout — how long a consumer may hold a message before it is redelivered. Too short redelivers work that was merely slow (duplicates); too long delays recovery from a genuinely dead consumer.
- Ack mode — auto-ack on receipt versus manual ack after processing. Auto-ack is faster but loses the message if the consumer dies mid-work; manual ack is the safe default for at-least-once.
- Redelivery limit — attempts before the message is dead-lettered. Higher rides out transient faults; lower fails fast to human review.
When it helps, and when it misleads¶
Its strength is that it is the workhorse of reliable messaging: it decouples sender and receiver in time — the receiver can be down when the message is sent — and guarantees the message outlives any single crash. Combined with idempotent consumers, at-least-once delivery yields effective exactly-once processing.
Its failure modes come from the shape of that guarantee. The guarantee is at-least-once, and the duplicates it implies are not optional — a consumer that assumes each message arrives exactly once will double-apply on every redelivery.[n1] Durability also has a cost people forget: fsync and replication add latency, and a deep durable backlog can quietly preserve and redeliver hours of work into a consumer that has been failing the whole time. The classic misuse is trusting the ack as proof of business success when the consumer acked too early — acknowledging on receipt rather than after the effect is durable means a crash loses the very message the queue swears it delivered. The discipline is to ack manually only after the effect is durable, treat idempotent consumers as a rule rather than an afterthought, and feed a bounded redelivery count into a dead-letter path so a poison message cannot loop forever.
How it implements the components¶
channel_delivery_semantics— it defines the delivery guarantee: persistent, at-least-once, redelivered-until-acknowledged, surviving both broker and consumer crashes.reply_and_receipt_policy— the acknowledgement is the receipt: the consumer's confirmation that flows back and controls whether the message is deleted or redelivered.
It guarantees transport but does not define the message's shape (message_contract — see Message Schema Registry), make reprocessing safe against the duplicates it creates (idempotency_and_correlation_marker — see Retry with Idempotency Key), or close the producer-side gap between committing state and emitting the message (the emit facet of channel_delivery_semantics — see Transactional Outbox/Inbox Relay). Its acknowledgement is the receipt facet of reply_and_receipt_policy; matching a substantive reply to a request is Request-Reply Correlation's.
Related¶
- Instantiates: Message-Mediated State Coordination — it is the reliable channel that lets participants exchange messages across time and failure without loss.
- Sibling mechanisms: Retry with Idempotency Key · Transactional Outbox/Inbox Relay · Bounded Mailbox or Queue · Backpressure Signal · Command Message Handler · Request-Reply Correlation
Editorial Notes¶
Form Classification¶
Form family: Control, Automation & Runtime
Rationale: Durable Queue with Acknowledgement operates as a live operational control that automatically routes, enforces, adapts, or responds during execution because it persists each message and keeps it until the consumer acknowledges success, redelivering on crash or timeout — so messages survive failure, at the cost of possible duplicates.
Independent corroboration: The frozen evidence defines Durable Queue with Acknowledgement as 'Persists each message and keeps it until the consumer acknowledges success, redelivering on crash or timeout — so messages survive failure, at the cost of possible duplicates', so its operative form is Control, Automation & Runtime.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Computer Science & Software Engineering
Origin pattern: Single lineage
Present-day reach: Specialized
Rationale: Distributed messaging cohered durable queues with acknowledgement, timeout, and redelivery to provide at-least-once delivery across consumer failure.
Review outcome: Independent reviewer agreement; high confidence.
Notes¶
The durable queue protects a message only once it is accepted. It says nothing about the producer's dual-write gap — the moment a participant commits a state change but crashes before enqueuing the message about it — which is Transactional Outbox/Inbox Relay's job. The durable queue is the reliable middle of the chain; making the entry to the queue atomic with the state change is a separate mechanism.
[n1] Over an unreliable channel, exactly-once delivery is impossible — you can guarantee at-most-once (may lose) or at-least-once (may duplicate), not both. Practical systems choose at-least-once and recover exactly-once processing by making consumers idempotent. ↩