Skip to content

TTL Expiration Sweep

An age-triggered staleness rule — instantiates Queue Draining

Automatically expires or revalidates queued items once they pass a defined time-to-live, so obsolete work stops dominating the drain — without becoming disguised load-shedding.

Some queued items simply stop being worth doing once they get old — a reservation nobody claimed, a request whose context has changed. TTL Expiration Sweep is the mechanical rule that acts on exactly that: it attaches a time-to-live to each item and periodically sweeps, applying a fixed disposition — hard-expire, or soft-expire via forced revalidation — to anything past the threshold. Its defining property is that time itself is the trigger: the sweep is stateless about an item's content, acting purely on its clock. That is what makes it cheap and fully automatable — and also what makes it dangerous when the threshold is set for convenience rather than real validity.

Example

An e-commerce checkout places a 15-minute hold on inventory when a shopper adds a limited item to their cart. Abandoned carts would otherwise lock stock indefinitely, starving buyers who are ready to pay. A TTL sweep runs continuously: any reservation older than its 15-minute TTL is expired and the stock returned to available. A gentler variant applies to pending-but-unpaid orders — at ≈30 minutes they are not silently deleted but flipped to revalidate: the shopper gets one prompt, and only a non-response expires the hold. Stock stays liquid, and a genuinely active shopper is never cut off mid-purchase by the clock alone.

How it works

  • Attach a clock to each item. Every item carries an age or a validity deadline against which it can be judged.
  • Sweep against the threshold. A periodic (or continuous) pass compares each item's clock to its TTL and selects everything over the line.
  • Apply a fixed disposition. Over-threshold items get one predetermined outcome — hard-expire, or soft-expire via forced revalidation — with the expiry logged.

Because it reads only the clock, never the content, it is trivially automatable; the whole art is in choosing the threshold and the hard-versus-soft disposition.

Tuning parameters

  • TTL length — the age threshold itself. Short keeps the queue fresh but expires slow-but-valid work; long is safe but lets stale items pile up and dominate the drain.
  • Hard expire versus soft revalidate — delete outright versus prompt for reconfirmation. Hard is cheap and decisive; soft is fairer but adds a step and a second wait.
  • Sweep cadence — continuous versus periodic. Continuous is precise but costly; periodic batches expirations, so items can outlive their TTL until the next pass.
  • Grace / reset policy — whether activity resets the clock. Resetting protects genuinely active items but can let a persistently-touched item never expire at all.

When it helps, and when it misleads

Its strength is keeping a drain focused on live work and reclaiming resources — inventory, locks, slots — that dead items were silently holding.

It misleads when the threshold is tuned by convenience rather than by real validity: then the sweep becomes disguised load-shedding, expiring valid obligations to make the queue smaller and calling it hygiene.[1] Hard-expiry with no revalidation step turns "old" into "gone" for items that were merely delayed. The classic misuse is shortening the TTL under backlog pressure to shrink the count — discarding work that a longer look would have honoured. The discipline is to set the threshold from the item's real validity horizon (when it genuinely stops being actionable), prefer revalidation over hard-expiry for anything consequential, and log expirations so the rule can be audited against true staleness rather than convenience.

How it implements the components

  • validity_and_staleness_rule — the age/validity threshold and the test for whether an item is still actionable.
  • disposition_path — the fixed outcome applied over the threshold: hard-expire, or soft-expire via forced revalidation.

It does not contact the people behind entries or offer freed slots (Appointment Waitlist Clearing), divert items that fail during processing rather than aging out (Dead-Letter Queue Processing), or decide when the overall drain is complete (Backlog Burn-Down).

  • Instantiates: Queue Draining — it drains the stale fraction of a queue by expiring or revalidating on age alone.
  • Sibling mechanisms: Appointment Waitlist Clearing · Dead-Letter Queue Processing · Incident Backlog Cleanup · Drain Dashboard · Backlog Burn-Down · Surge Worker Pool · Graceful Queue Shutdown · Connection Draining · Maintenance Drain · Message Queue Drain

References

[1] Load shedding — deliberately dropping or rejecting work to protect a system — is a distinct, legitimate pattern, but a TTL sweep is not supposed to be it. The line between expiring genuinely dead items and shedding valid load under pressure is exactly what the threshold and the audit protect. ↩