Skip to content

Connection Draining

A drain protocol at a traffic boundary — instantiates Queue Draining

Takes a server out of the load balancer's rotation and lets its in-flight requests finish — up to a hard timeout — before the instance is stopped.

When an instance behind a load balancer must be removed — a deploy, a scale-in, a failing host — killing it outright drops whatever requests it was mid-way through serving. Connection Draining is the protocol that avoids that: it deregisters the endpoint so the balancer stops assigning it new connections, lets the in-flight ones run to completion, and force-closes any stragglers at a defined timeout so the removal always terminates. Its defining idea is a bounded grace period at the traffic-assignment boundary: no new work arrives, existing work is allowed to finish, and a hard cutoff guarantees the drain can't be held hostage by one hung connection.

Example

A web fleet of a dozen instances sits behind a load balancer, and a rolling deploy needs to replace them one at a time. To retire an instance, it is marked draining. The balancer immediately stops routing new connections to it, but the ≈30 requests already open — page loads, API calls — keep being served. Over the next ≈45 seconds they complete and the connection count falls to near zero. A single slow file upload is still open at the 120-second drain timeout, so it is force-closed and the instance is stopped anyway; the deploy is not stalled by one straggler. The user-visible result across the whole rollout is zero dropped requests and a bounded, predictable wait per instance.

How it works

The protocol is deliberately narrow — it does not reorder or triage anything, it just bounds a wait:

  • Deregister from new assignment. The endpoint is marked draining; the balancer stops sending it new connections while keeping the existing ones alive.
  • Let in-flight complete. Open requests run to their natural finish — the drain's success condition is "no in-flight connections remain."
  • Cut off at a timeout. A drain timeout force-closes any connection still open past the limit, so the drain always terminates and the deploy proceeds.

Tuning parameters

  • Drain timeout — how long to wait for in-flight work before force-closing. Longer preserves more requests but slows every deploy and lets one hung connection stall a rollout; shorter is snappier but abandons more mid-flight work.
  • Health-check dwell — how quickly the balancer actually stops routing after the draining mark. Too slow keeps handing the instance new connections it will have to abandon.
  • Force-close behaviour — hard reset versus graceful TCP close versus returning a "retry elsewhere" response to the client on cutoff.
  • Connection-type scope — treating long-lived connections (WebSockets, streams) differently from short HTTP requests, which finish on their own within any sane timeout.

When it helps, and when it misleads

Its strength is zero-downtime deploys and scale-in: users never hit a killed instance, and the timeout guarantees the drain is finite rather than open-ended.

It misleads in two ways. Long-lived connections may never finish inside any reasonable timeout, so for them "draining" silently degrades to force-close — a fact easy to miss until a streaming feature starts dropping mid-session. And a too-generous timeout lets one stuck request hold an entire rollout hostage. The classic misuse is trusting connection draining to protect stateful work that isn't actually request-scoped: it finishes the HTTP request but not a half-written database transaction the request kicked off. The discipline is to set the timeout from real request-duration percentiles and to treat force-close as a first-class outcome to design for, not an edge case.[1]

How it implements the components

  • admission_pause — deregistering the endpoint from new-connection assignment is the intake pause, scoped to one instance.
  • completion_criteria — the drain is done when in-flight connections reach zero.
  • safety_cutoff — the drain timeout force-closes stragglers so removal always terminates.

It does not decide which queued items deserve to finish or persist the unfinished ones (Graceful Queue Shutdown), nor order how a pool of consumers works through a backlog (Message Queue Drain).

  • Instantiates: Queue Draining — it drains in-flight requests off one endpoint at the traffic boundary.
  • Sibling mechanisms: Graceful Queue Shutdown · Maintenance Drain · Message Queue Drain · Drain Dashboard · Surge Worker Pool · Backlog Burn-Down · Incident Backlog Cleanup · Dead-Letter Queue Processing · TTL Expiration Sweep · Appointment Waitlist Clearing

Notes

Connection draining protects request-scoped work only. Anything the request set in motion beyond its own lifetime — an async job, an uncommitted transaction — needs its own durability path; the drain finishing the connection is not the same as the side effects completing.

References

[1] Real load balancers implement this directly — AWS calls it connection draining / deregistration delay, and equivalents exist across Envoy, NGINX, and Kubernetes readiness gates. The named parameter there is exactly the drain timeout described above. ↩