Binary entropy function¶

The Shannon entropy of a Bernoulli variable as a concave function of its success probability.

Version: v2 · 2026-09-19 · History
Domain-specific #: 3463
Domain group: Formal Sciences
Origin domain: Information Theory

Core Idea¶

For probability p and a fixed logarithm base, binary entropy is minus p log p minus one-minus-p log one-minus-p, with zero-log-zero defined by continuity. Expected self-information from the two outcomes produces a symmetric concave curve, zero at deterministic endpoints and maximal at the unbiased probability. The abstraction is therefore identified by a declared carrier, a transformation or constraint over that carrier, and an invariant that tells an analyst whether the named structure is genuinely present.

The load-bearing residual is not the broad topic of information theory. It is the domain-specific identity determined by p lies in the unit interval, the log base fixes the information unit, endpoint limits are used, symmetry holds under p versus one-minus-p, and the maximum occurs at one half.

How would you explain it like I'm…

The Coin Surprise Meter

Think about flipping a coin. If the coin always lands heads, you are never surprised. If it lands heads or tails equally often, you can never guess, so it surprises you the most. The binary entropy function is a number that says how surprising a two-way choice is on average.

Two-Choice Surprise Hill

Some events have only two outcomes, like heads or tails, and one outcome happens with some chance p. The Binary entropy function measures how uncertain you are, on average, before you see the result. If p is 0 or 1, you already know the answer, so the uncertainty is zero. It is biggest when both outcomes are equally likely, at p = one half. It also treats the two outcomes the same way, so a coin that lands heads 90% of the time is just as uncertain as one that lands tails 90% of the time.

Two-Outcome Uncertainty Curve

The binary entropy function gives the average information from an event with two outcomes of probability p and 1 - p. Its formula is H(p) = -p log p - (1 - p) log(1 - p), where the base of the logarithm sets the unit, such as bits for base 2. At p = 0 or p = 1, the terms 0 log 0 are taken to be zero, so H is zero when the result is certain. The curve is symmetric, since H(p) = H(1 - p), and it is concave, reaching its maximum at p = 1/2. It is the expected surprise of a biased coin, not the surprise of any single flip.

The Binary entropy function is the Shannon entropy of a two-outcome (Bernoulli) random variable, written as a function of the probability p in [0, 1]: H(p) = -p log p - (1 - p) log(1 - p). It is the expected self-information of the two outcomes. The logarithm base fixes the unit of information, for example bits for base 2 and nats for base e. The endpoints are handled by the continuity convention 0 log 0 = 0, so H(0) = H(1) = 0, reflecting that a deterministic outcome carries no uncertainty. The function is symmetric, H(p) = H(1 - p), concave on the unit interval, and attains its unique maximum at p = 1/2. Using it correctly means checking that the argument really is a probability of one of two outcomes and stating the log base, rather than applying the formula to anything two-valued by resemblance.

Scope of Application¶

Binary entropy function belongs to information theory and is useful where the analyst can specify the typed information theory carrier, defining objects and relations, parameters, conventions, evidence, boundary cases, and comparison targets, then evaluate p lies in the unit interval, the log base fixes the information unit, endpoint limits are used, symmetry holds under p versus one-minus-p, and the maximum occurs at one half. The scope is broad within that domain but bounded by the need for p lies in the unit interval, the log base fixes the information unit, endpoint limits are used, symmetry holds under p versus one-minus-p, and the maximum occurs at one half.

Clarity¶

The abstraction clarifies a crowded vocabulary by making p lies in the unit interval, the log base fixes the information unit, endpoint limits are used, symmetry holds under p versus one-minus-p, and the maximum occurs at one half the center of the account. A claim should name the carrier, the governing operation or relation, the applicable assumptions, and the recognition test.

Manages Complexity¶

Without the abstraction, an analyst must reason directly over many local details: the carrier roles, admissibility assumptions, competing conventions, derived invariants, boundary cases, and proof or validation obligations specific to Binary entropy function. Binary entropy function compresses them into the roles in the structural signature. That compression permits comparison across instances without erasing the variables that determine validity. It also exposes which details may be varied safely and which are constitutive.

Abstract Reasoning¶

Identify the carrier. State what the elements, states, objects, or observations are: the typed information theory carrier, defining objects and relations, parameters, conventions, evidence, boundary cases, and comparison targets. Reject examples whose alleged carrier belongs to a different problem.
Lock the constitutive rule. Express p lies in the unit interval, the log base fixes the information unit, endpoint limits are used, symmetry holds under p versus one-minus-p, and the maximum occurs at one half independently of one notation or implementation.

Knowledge Transfer¶

Knowledge transfers strongly among subfields of information theory because they reuse the typed information theory carrier, defining objects and relations, parameters, conventions, evidence, boundary cases, and comparison targets, Expected self-information from the two outcomes produces a symmetric concave curve, zero at deterministic endpoints and maximal at the unbiased probability., and type the carrier, state every parameter and convention in the definition, test that p lies in the unit interval, the log base fixes the information unit, endpoint limits are used, symmetry holds under p versus one-minus-p, and the maximum occurs at one half, compare the nearest accepted identity, and report counterexamples, uncertainty, and limiting cases.

Relationships to Other Abstractions¶

Current abstraction Binary entropy function Domain-specific

Parents (1) — outgoing typed relations

Binary entropy function is a kind of Uncertainty Prime

What makes it its own entry: the domain-specific identity determined by p lies in the unit interval, the log base fixes the information unit, endpoint limits are used, symmetry holds under p versus one-minus-p, and the maximum occurs at one half.

Hierarchy path (1) — routes to 1 parentless root

Binary entropy function → is a kind of Uncertainty

Neighborhood in Abstraction Space¶

Binary entropy function sits in a crowded region of the domain-specific corpus (11^th percentile for distinctiveness): several abstractions share nearly its structure, so a description that fits it tends to fit its neighbors too.

Family — Information Theory & Error-Correcting Codes (20 abstractions)

Nearest neighbors

Min-entropy — 0.94
Information dimension — 0.94
Asymptotic equipartition property — 0.92
Typical set — 0.92
Directed information — 0.91

Computed from structural-signature embeddings · 2026-10-25