Tensions in Practice: Low average payload in tension with a fixed-size message slot¶
An invented binary message protocol with shared context
Sender and receiver already share the current mode. In Known mode, which occurs 90% of the time, the target is already determined. In Open mode, occurring 10%, it is equally likely to be one of eight values and needs three binary digits to specify. Residual uncertainty averages 0.9 × 0 + 0.1 × 3 = 0.3 bits. A variable payload can use that average in this toy protocol, but every Open message still needs three bits.
Spend bits only when needed
Use the shared mode to omit determined payloads.
Keep one fixed message slot
Reserve the same payload length for every message.
Why these aims pull against each other
A small expectation does not bound the largest per-message requirement. Fixed slots simplify length handling but spend space in already-determined contexts.
Choose an arrangement to see what changes and what remains difficult.
The mode frequencies and residual uncertainty stay fixed. Only reserved payload changes; the rare three-bit requirement remains visible in both arrangements.
What this choice protects
What it costs
When it fits
Compare the arrangements
Use mode-dependent payloads
Send no target bits in Known mode and three in Open mode; the shared mode tells the receiver how many to expect.
| Chance | Bits left | Bits sent | |
|---|---|---|---|
| Known mode | 0.9 | 0 | 0 |
| Open mode | 0.1 | 3 | 3 |
| Average | — | 0.3 | 0.3 |
- What it protects
- Mean target payload is 0.3 bits per message under the stipulated law.
- What it costs
- Individual payload demand still reaches three bits, and both ends must use synchronized mode information.
- When it fits
- Fits a variable-length transport with shared mode and boundaries already available.
Illustration note: The calculation counts target payload only. It does not make framing, timing, shared context or synchronization free.
Reserve three bits each time
Use a three-bit payload slot in both modes, padding the already-known target when needed.
| Chance | Bits left | Bits sent | |
|---|---|---|---|
| Known mode | 0.9 | 0 | 3 padded |
| Open mode | 0.1 | 3 | 3 |
| Average | — | 0.3 | 3 |
- What it protects
- Payload offsets and slot capacity stay constant without a mode-dependent length parser.
- What it costs
- Known messages consume three reserved bits despite zero remaining target uncertainty.
- When it fits
- Fits a fixed-record consumer whose simpler layout is worth the unused information capacity.
Illustration note: Residual entropy is still 0.3 bits; fixed storage length is a protocol choice, not a changed probability law.
What this illustration does—and does not—establish
The source supplies the structural tension; the invented example makes one relation inspectable. Costs and conditions are part of each arrangement, not exceptions to a universal recommendation.
- The shared mode, message boundaries and exact distribution are stipulated. Their communication cost is outside the displayed payload.
- This does not claim entropy equals a realized error rate, semantic importance or arbitrary codec size.
- Both arrangements serve every Open message; neither budgets only the average and silently drops rare messages.
Source entries
Conditional Entropy
The canonical tension motivates this comparison. The setting, finite values and arrangements are declared editorial illustrations, not measured findings.
average versus tail
T2: average versus tail. Low average residual uncertainty can coexist with catastrophic ambiguity in rare contexts. Diagnostic: inspect \(H(Y\mid X=x)\), class-conditional errors, and high-consequence strata rather than relying only on the expectation.
The source operation
Conditional Entropy measures the expected uncertainty that remains about a target after specified context is known. For discrete random variables, let (Y) be the target and (X) the context. Each possible context value (x) changes the probability distribution of (Y). Compute the Shannon entropy of each resulting conditional distribution and average those entropies according to how often each context occurs. The result is \(H(Y\mid X)\): a single quantity for the unresolved information about (Y) after observing (X).