Rate Limit with Burst Allowance¶

Inflow-governing protocol — instantiates Intermittent Burst Absorption

Lets inflow spike freely up to a bounded allowance but caps the sustained rate, so short bursts pass while a prolonged flood is throttled or shed before it exhausts the system.

A Rate Limit with Burst Allowance governs inflow with a deliberate two-part shape: a generous allowance for short spikes and a firm ceiling on the sustained rate. It welcomes the brief burst — the whole point is not to punish momentary spikes — while ensuring that a prolonged flood, the kind that would drain buffers and leave no room to recover, gets throttled or shed at the door. What makes it this mechanism is that it acts on the input side and distinguishes bursts by duration: it draws the line between "a short spike the system can absorb" and "a sustained overload it cannot," and it enforces that line by capping or rejecting what exceeds the envelope, rather than holding excess or adding capacity to serve it.

Example¶

A freeway on-ramp feeds a highway that flows smoothly at moderate volume but seizes into stop-and-go once inflow is sustained above what the mainline can carry. Left alone, a rush of cars all merging at once collapses the whole highway for everyone. So the transportation authority installs ramp metering: a signal at the ramp that releases cars onto the highway at a controlled rate. It's engineered with a burst allowance — a short platoon of cars can go through on a green without much delay — but it caps the sustained rate of entry to what the mainline can absorb, holding back the rest at the ramp. A brief surge passes; a prolonged flood is metered down so it never tips the highway into gridlock. The mainline keeps flowing, which means the ramp queue itself drains faster than the alternative of a jammed highway. The limiter didn't add lanes or store cars long-term; it shaped the input so short bursts pass and sustained overload is held out.

How it works¶

The distinguishing content is the token-bucket shape — an allowance that refills at the sustained cap:

Set a sustained rate the system can carry indefinitely without draining its recovery capacity — the long-run ceiling on inflow.
Grant a burst allowance above that rate, a bounded budget that lets short spikes through instantly; it refills at the sustained rate, so it's available again after a quiet stretch.^[1]
Enforce at the edge when the allowance is spent — throttle, delay, or reject the excess, so a prolonged flood is shaped down before it reaches and overwhelms downstream capacity.
Signal the limit to well-behaved senders (a "retry after" or slow-down cue) so they back off rather than hammer the door.

Tuning parameters¶

Sustained rate — the long-run ceiling. Higher serves more but risks the downstream and leaves less recovery headroom; lower is safer but rejects more legitimate load.
Burst allowance size — how large a spike passes untouched. Bigger welcomes real bursts but lets more through before the cap engages; smaller is protective but clips genuine spikes.
Allowance refill rate — how fast the burst budget replenishes, setting how soon back-to-back bursts are each allowed.
Overflow action — what happens to excess once the allowance is spent: queue briefly, throttle, or hard-reject, trading latency against outright loss.
Enforcement granularity — per-sender or global. Per-sender contains one heavy source fairly but costs more to track; global is simple but lets one sender crowd out others.

When it helps, and when it misleads¶

Its strength is protecting the system's ability to recover: by capping sustained inflow while still welcoming short spikes, it keeps a prolonged flood from draining the buffers and reserves that every other absorption mechanism depends on — it is the guard that keeps a burst from becoming a permanent overload.

The failure modes come from the two settings. Too tight, it becomes a self-inflicted outage — clipping legitimate bursts the system could easily have absorbed, and turning a good day (a viral moment, a real rush) into rejected users. Too loose, it lets a sustained flood through and provides no protection at all. A subtle one: when the limit rejects requests, naive clients often retry immediately and in unison, amplifying the very burst the limiter was meant to tame. The classic misuse is deploying it purely as a cost or gatekeeping control — throttling to shift load onto users — and calling it burst protection. The discipline is to set the sustained rate to genuine recovery capacity, size the allowance to real bursts, signal backoff so clients don't retry-storm, and revisit both as observed bursts show what "short" and "sustained" actually look like.

How it implements the components¶

burst_threshold — the sustained rate and burst allowance together are the explicit boundary between ordinary handling and burst-mode limiting, expressed on the inflow.
degradation_or_shedding_gate — once the allowance is spent, throttling or rejecting the excess is the deliberate shedding that keeps a prolonged flood from exceeding the safe envelope.

It does not hold what it declines to serve — a Burst Buffer or Surge Queue does that — nor decide *which items are worth serving (Triage Protocol); it caps and sheds by rate at the input.*

Instantiates: Intermittent Burst Absorption — it shapes inflow so short bursts pass and sustained overload is shed before it exhausts the system.
Sibling mechanisms: Triage Protocol · Burst Buffer · Surge Queue · Elastic Capacity Scaling · Overflow Channel · Peak Response Reserve · Backup Staffing Pool · On-Call Response Rotation · Incident Surge Team · Flash Crowd Playbook · Post-Burst After-Action Review

Notes¶

Shedding by rate is blunt: past the allowance, this protocol drops or delays whatever arrives, regardless of importance. When some of the excess is far more valuable than the rest, pair it with a Triage Protocol so what gets shed is the least critical load, not merely the last to arrive.

References¶

[1] The token bucket is the classic algorithm for exactly this shape: tokens accrue at the sustained rate up to a bucket size (the burst allowance), each admitted unit spends a token, and requests are throttled or dropped when the bucket is empty. It is the standard, correctly-used realization of "allow short bursts, cap the sustained rate." ↩