Skip to content

Exponential Backoff with Jitter

Retry protocol — instantiates Synchronized Release Dampening

Turns a retry storm into a decorrelated trickle by making each rejected caller wait an exponentially growing, randomly perturbed delay before trying again.

Exponential Backoff with Jitter governs the retry path: when a caller is rejected or times out, it waits before retrying, and each successive failure roughly doubles the wait — with a random offset added to every delay. The doubling drains total retry pressure as failures persist; the random offset is the part that matters most here, because it is what stops the failed callers from re-colliding in lockstep. A fleet that retries on a fixed timer stays phase-aligned forever — it just re-hits the choke point on a metronome. The defining move of this mechanism is that it is reactive and self-decorrelating: it triggers only after a failure, and the jitter smears the survivors across a widening window so the next wave is never as sharp as the last.

Example

A card-authorization endpoint briefly returns 503s during a database failover, and roughly 40,000 point-of-sale terminals in the field all fail their auth call at once. Every terminal retries on a fixed one-second loop, so the endpoint — which is trying to recover — gets hammered by a fresh synchronized wave every second and never gets clear air to drain its backlog. Swapping in exponential backoff with full jitter changes the shape of the traffic: after its first failure a terminal waits a random time in [0, 1s], after the second in [0, 2s], then [0, 4s], and so on up to a cap. The retries stop arriving as pulses and start arriving as a smear; total attempts fall as the outage drags on, and the endpoint finally sees enough quiet to recover, at which point successful calls reset each terminal's delay to zero.

How it works

The two dials do two different jobs, and both are needed:

  • Exponential growth of the delay ceiling reduces the volume of retries while the resource is down — persistence is penalized, so a prolonged outage is not met with undiminished hammering.
  • Jitter — a random draw within that ceiling — reduces the correlation of retries. Under "full jitter" each caller picks a uniform random point in [0, ceiling], which maximally scatters them; "equal jitter" keeps a fixed half plus a random half; no jitter leaves them synchronized.

The ceiling is capped so waits do not grow without bound, attempts are bounded by a retry budget, and a success resets the schedule.

Tuning parameters

  • Base delay and multiplier — the starting wait and how fast it grows; steeper backoff clears a struggling resource faster but lengthens the tail latency for legitimate callers.
  • Jitter mode — none, equal, or full; full jitter decorrelates best but makes any single caller's timing least predictable.
  • Maximum cap — the ceiling the delay stops doubling at; too low and the fleet stays synchronized near the cap, too high and stragglers wait absurdly long.
  • Retry budget — how many attempts before giving up; a shared budget can cause a synchronized give-up if everyone exhausts it at once.
  • Reset policy — whether one success fully resets the schedule or decays it gradually.

When it helps, and when it misleads

Its strength is that it needs no coordination and no server-side state: each caller decorrelates itself locally, which is what makes it the default cure for retry storms and cache-stampede re-collisions. Its signature failure mode is synchronized jitter — if every client seeds its randomness the same way, uses the same fixed window, or runs pure exponential backoff with no jitter, the callers stay phase-aligned and the storm survives the "fix."[1] Backoff set too aggressively swings the other way, producing long tail latency or a synchronized give-up as the whole fleet exhausts its budget together. The classic misuse is cranking the multiplier to shield the server while silently starving latency-critical calls. The discipline is to use full jitter, cap the delay, bound attempts, and route anything urgent to an escape lane rather than leaving it to back off.

How it implements the components

  • retry_backoff_contract — it is the contract: the per-attempt rule that sets each successive delay from the failure count.
  • jitter_budget — the random window added to every delay; the term that actually decorrelates the retriers rather than merely slowing them.
  • dispersion_policy — the net effect is a dispersion policy for the retry path, spreading attempts across an expanding window.

It does not bound how many concurrent callers the choke admits — that admission cap is Capacity-Aware Reconnect Queue — nor detect when the resource has recovered (Half-Open Circuit Probe), nor partition the population into planned waves (Cohort-Based Reactivation).

References

[1] Full jitter is a standard, well-documented variant of exponential backoff in which the delay is drawn uniformly from the whole [0, ceiling] interval rather than added on top of a fixed base. It is named here as a real technique because it is the specific corrective for the thundering herd problem this mechanism otherwise risks re-creating: backoff without jitter reduces load but not correlation. ↩