Tensions in Practice: Setup sharing in tension with sparse-arrival waiting¶
Grouped processing · sparse arrivals
Grouping items lets one setup serve several pieces of work. But a rule that waits only for a full batch has a hidden dependency: another arrival must release the items already waiting. A time backstop removes that dependency, at the cost of sometimes paying the setup for a small group.
Share setup cost
Amortize a real fixed setup across enough items to make grouping worthwhile.
Release sparse work
Keep a few remaining items from waiting indefinitely for more arrivals.
Why these aims pull against each other
A size trigger protects group size but depends on future arrivals. A time trigger protects release opportunity but cannot promise an economical batch size.
Choose an arrangement to see what changes and what remains difficult.
Qualitative paths and conditions, not measured costs, timings or performance guarantees.
What this choice protects
What it costs
When it fits
Compare the arrangements
Size only
Release the batch only after it contains the chosen number of items.
- What it protects
- Each release shares setup across the intended group size.
- What it costs
- A partial final batch can wait indefinitely if no more items arrive.
- When it fits
- Arrivals reliably fill batches and there is no independent turnaround requirement for the sparse tail.
Illustration note: This option is deliberately conditional, not a general recommendation. A stopped stream exposes the missing release event.
Size or time
Release when either the target size is reached or the waiting limit expires.
- What it protects
- Sparse items no longer require another arrival to become eligible for processing.
- What it costs
- Small partial groups repeat the fixed setup more often per item.
- When it fits
- Turnaround matters under light load and the system can afford less setup amortization.
Illustration note: The timer bounds eligibility to start, not end-to-end completion; scheduler delays, processing time, and failures remain outside the sketch.
What this illustration does—and does not—establish
Batch Processing: Flush Rule versus Latency Bound (temporal) supplies the release-trigger conflict; Batch Processing: Per-Batch Setup versus Per-Item Cost (scalar) supplies the real setup-cost precondition.
- Batching helps only when there is a real fixed setup worth sharing.
- No timer duration, batch size, or throughput guarantee is supplied.
Source entries
Batch Processing
Batch Processing: Flush Rule versus Latency Bound (temporal) supplies the conflict examined here.
Flush Rule versus Latency Bound (temporal)
T5 — Flush Rule versus Latency Bound (temporal). Every batch system needs an explicit flush rule (size-, time-, pressure-, or trigger-based) and the choice fixes the latency profile — a size-based flush can stall the last items indefinitely if the batch never fills. The boundary is the release condition. The failure mode is a pure size-based flush under light load, where a half-full batch waits forever and per-item latency becomes unbounded. Diagnostic: under the lightest expected arrival rate, what is the worst-case wait for an item under this flush rule? A size-only rule needs a time-based backstop, or sparse arrivals starve in the buffer.
Per-Batch Setup versus Per-Item Cost (scalar)
T2 — Per-Batch Setup versus Per-Item Cost (scalar). Batching pays only when the fixed per-batch cost is large relative to the per-item cost; absent that asymmetry there is nothing to amortise. The boundary is the cost ratio. The failure mode is batching where the setup is already cheap (no gain, only added latency) or refusing to batch where the setup dominates (paying the fixed cost on every item). Diagnostic: is there a real fixed cost — setup, warm-up, context-switch, cognitive ramp — incurred per batch independent of size? If the per-item cost dominates, batching buys latency for nothing; the asymmetry is the precondition.