Skip to content

Weighted Fair Queue

Scheduling protocol — instantiates Request–Response Capability Provisioning

Serves competing requests in an order that gives each client or class a guaranteed share of capacity, so no stream is starved and none can monopolize the server.

When more requests arrive than a shared capability can serve at once, something has to decide who goes next. The Weighted Fair Queue answers "next" by allocating each client or traffic class a configured share of the server's throughput, then interleaving their requests to honor those shares over time. Its defining move is bounded fairness under contention: a heavy client cannot crowd out a light one, and a low-priority stream still makes progress, because the queue serves in proportion to assigned weights rather than by first-come-first-served or strict priority. It is purely an ordering policy — it neither admits nor rejects load, and it neither tracks nor measures it; it only decides the sequence among requests already accepted.

Example

A university's shared GPU cluster of 64 cards serves three groups: a genomics lab (weight 5), a climate-modelling group (weight 3), and ad-hoc student coursework (weight 2). At deadline season all three flood the queue at once. The fair-queue scheduler interleaves their jobs so that, over the busy window, genomics receives roughly 50% of GPU-time, climate ~30%, and coursework ~20% — regardless of who happened to submit a 500-job burst first. A single student scripting hundreds of tiny jobs can no longer freeze out the genomics lab's long run; conversely, coursework never starves to zero even while the big labs are saturated. The same hardware now feels fair to everyone, and contention is resolved by a published policy rather than by whoever writes the tightest submission loop.

How it works

  • Assign weights and classes. Each client or service class gets a share of capacity, not a fixed slot, so idle share flows to whoever is active.
  • Track service already received. The queue remembers how much each class has recently been served (via virtual-time or deficit counters), so it can favor whoever is furthest behind its entitlement.
  • Interleave to honor shares. Dispatch the next request from the most under-served class, approximating max-min fairness across the classes.
  • Degrade proportionally. When everyone is over-subscribed, shares shrink in proportion rather than some class collapsing to nothing.

Tuning parameters

  • Weights — the relative share each class receives. The core dial and the most political: weights encode whose work matters, and mis-set weights entrench the loudest rather than the most important.
  • Class granularity — per-user, per-team, or per-request-type. Fine classes give precise fairness but multiply bookkeeping and invite clients to split identities to grab more shares.
  • Quantum / burst size — how much work a class may do in one turn before yielding. Large quanta cut switching overhead but coarsen fairness at short timescales.
  • Starvation floor — a guaranteed minimum share for even the lowest class. Raising it protects small clients but caps how much a hot class can borrow when others are idle.
  • Preemption — whether an in-flight long request can be paused for a starved class. Preemption tightens fairness but wastes partial work and adds complexity.

When it helps, and when it misleads

Its strength is that it converts contention from a race into a policy: every class is guaranteed progress, no client can monopolize the server, and the allocation is explicit and auditable rather than emergent from timing luck.

Its limitation is that it optimizes share, not outcome. A perfectly fair split of an undersized server still fails everyone — the queue can only ration capacity, never create it — so fair scheduling on a starved service just distributes the pain evenly and can hide the real need for more capacity. The classic misuse is using the weights as a covert priority lever — quietly bumping the executive team's share while still calling the result "fair" — or slicing classes so finely that clients fragment into many identities to game their allocation. The discipline that keeps it honest is to publish the weights and the fairness criterion,[1] review them openly, and pair the queue with a capacity mechanism, since a scheduler can only divide what already exists.

How it implements the components

The fair queue realizes the ordering side of the archetype's contention machinery — nothing more:

  • multiplexing_and_scheduling_policy — the rule that interleaves many clients' requests onto one capability and picks what runs next.
  • priority_service_class_policy — the weights and classes that encode each stream's entitled share of throughput.

It does not decide whether a request is admitted in the first place (that boundary is Rate Limit with Burst Allowance), add the capacity it rations (Autoscaling Worker Pool), or report how long anyone waited (Service-Level Monitor); the fair queue only orders requests that have already been accepted.

Notes

A fair queue rations capacity but cannot manufacture it. On a chronically undersized server, fairness merely spreads the shortfall evenly across classes — which can mask the underlying need for more capacity by making a starved service feel equitable instead of visibly broken. It belongs alongside a capacity or admission mechanism, not in place of one.

References

[1] Max-min fairness — an allocation in which no class's share can be increased without decreasing the share of some already-smaller class. It is the fairness criterion weighted fair queuing approximates, and stating it explicitly is what lets clients judge whether the weights are just. ↩