Directed information

An information-theoretic quantity summing the conditional information that the present and past of one process provide about each current value of another given the latter's past.

Version

v2 · 2026-09-19 · History

Domain-specific #

4191

Domain group

Formal Sciences

Origin domain

Information Theory

Core Idea

Massey's directed information I(X^n to Y^n)=sum I(X^i;Y_i|Y(i-1)) distinguishes causal temporal direction and feedback from symmetric mutual information. At each time, conditioning removes what the destination's own history already predicts, then mutual information measures the additional contribution available from the source history; summation accumulates the flow. The abstraction is therefore identified by a declared carrier, a transformation or constraint over that carrier, and an invariant that tells an analyst whether the named structure is genuinely present.

How would you explain it like I'm…

One-Way Guessing Help

Directed information measures how much one friend's past words help you guess what another friend will say next — beyond what you could already guess from the second friend's own past words. It goes one way, so how much A helps predict B can be different from how much B helps predict A.

One-Way Information Flow

Directed information is a way of measuring how much one stream of signals helps predict another, going in one direction through time. At each moment, you first see how well the receiver's own past predicts its next value. Then you check how much extra the sender's past and present add on top of that. You add up those extra bits over all the moments. Unlike ordinary shared information, which is the same both ways, directed information from A to B can be different from B to A, which helps spot feedback.

Causal Information Flow

Directed information, defined by Massey, measures how much information flows from one random process X to another process Y over time, respecting the order in which things happen. Ordinary mutual information measures how much two things share and is symmetric: the same from X to Y as from Y to X. Directed information is not symmetric. At each time step i, it asks how much X's history up to i tells you about Y at time i beyond what Y's own past already predicts, which is a conditional mutual information. Adding those amounts over all time steps gives the total, I(Xⁿ → Yⁿ) = Σ I(Xⁱ; Yᵢ | Yⁱ⁻¹). This makes it useful for systems with feedback, where Y can also influence later X.

 

Directed information, introduced by Massey, is defined for paired stochastic processes over a horizon n as I(X^n → Y^n) = Σ_{i=1}^{n} I(X^i; Y_i | Y^{i−1}), where X^i denotes the source history up to time i and Y^{i−1} the destination's past. At each time step, conditioning on Y^{i−1} removes whatever the destination's own history already predicts about Y_i, and the conditional mutual information then measures the additional contribution available from the source history; the sum accumulates this causal flow over the horizon. Unlike mutual information I(X^n; Y^n), which is symmetric, directed information respects temporal order, and in general I(X^n → Y^n) ≠ I(Y^n → X^n). This makes it the appropriate quantity for settings with feedback, where the source may depend on past outputs. Using it correctly requires specifying the processes, the time horizon, the causal conditioning convention (for example whether X_i is included alongside Y_i's past) and the joint distribution, with the conditional-mutual-information sum finite.

Scope of Application

Directed information belongs to information theory and is useful where the analyst can specify the typed information theory carrier, defining objects and relations, parameters, conventions, evidence, boundary cases, and comparison targets, then evaluate the paired stochastic processes, time horizon, causal conditioning convention, joint distribution and finite conditional-mutual-information sum are explicit. The scope is broad within that domain but bounded by the need for the paired stochastic processes, time horizon, causal conditioning convention, joint distribution and finite conditional-mutual-information sum are explicit. The entry records a descriptive analytical identity; practical use requires the governing domain's evidence, standards, and safety obligations.

Clarity

The abstraction clarifies a crowded vocabulary by making the paired stochastic processes, time horizon, causal conditioning convention, joint distribution and finite conditional-mutual-information sum are explicit the center of the account. A claim should name the carrier, the governing operation or relation, the applicable assumptions, and the recognition test. A bare label is insufficient because the name Directed information can be used for a formal identity, an implementation, or a neighboring result unless carrier and convention are stated.

Manages Complexity

Without the abstraction, an analyst must reason directly over many local details: the carrier roles, admissibility assumptions, competing conventions, derived invariants, boundary cases, and proof or validation obligations specific to Directed information. Directed information compresses them into the roles in the structural signature. That compression permits comparison across instances without erasing the variables that determine validity. It also exposes which details may be varied safely and which are constitutive.

Abstract Reasoning

1. Identify the carrier. State what the elements, states, objects, or observations are: the typed information theory carrier, defining objects and relations, parameters, conventions, evidence, boundary cases, and comparison targets. Reject examples whose alleged carrier belongs to a different problem.
2. Lock the constitutive rule. Express the paired stochastic processes, time horizon, causal conditioning convention, joint distribution and finite conditional-mutual-information sum are explicit independently of one notation or implementation.

Knowledge Transfer

Knowledge transfers strongly among subfields of information theory because they reuse the typed information theory carrier, defining objects and relations, parameters, conventions, evidence, boundary cases, and comparison targets, At each time, conditioning removes what the destination's own history already predicts, then mutual information measures the additional contribution available from the source history; summation accumulates the flow., and type the carrier, state every parameter and convention in the definition, test that the paired stochastic processes, time horizon, causal conditioning convention, joint distribution and finite conditional-mutual-information sum are explicit, compare the nearest accepted identity, and report counterexamples, uncertainty, and limiting cases.

Relationships to Other Abstractions

Current abstraction Directed information Domain-specific

Parents (1) — outgoing typed relations

• Directed information is a kind of Information Prime

  It remains its own entry because its identity is fixed by the paired stochastic processes, time horizon, causal conditioning convention, joint distribution and finite conditional-mutual-information sum.

Hierarchy path (1) — routes to 1 parentless root

• Directed information → is a kind of Information → is a kind of Uncertainty

Neighborhood in Abstraction Space

Directed information sits in a crowded region of the domain-specific corpus (8^th percentile for distinctiveness): several abstractions share nearly its structure, so a description that fits it tends to fit its neighbors too.

Family — Information Theory & Error-Correcting Codes (20 abstractions)

Nearest neighbors

• Min-entropy — 0.93
• Information dimension — 0.93
• Asymptotic equipartition property — 0.93
• Typical set — 0.92
• Binary entropy function — 0.91

Computed from structural-signature embeddings · 2026-10-25