Directed information¶
An information-theoretic quantity summing the conditional information that the present and past of one process provide about each current value of another given the latter's past.
Core Idea¶
Massey's directed information I(X^n to Y^n)=sum I(Xi;Y_i|Y(i-1)) distinguishes causal temporal direction and feedback from symmetric mutual information. At each time, conditioning removes what the destination's own history already predicts, then mutual information measures the additional contribution available from the source history; summation accumulates the flow. The abstraction is therefore identified by a declared carrier, a transformation or constraint over that carrier, and an invariant that tells an analyst whether the named structure is genuinely present.
Scope of Application¶
Directed information belongs to information theory and is useful where the analyst can specify the typed information theory carrier, defining objects and relations, parameters, conventions, evidence, boundary cases, and comparison targets, then evaluate the paired stochastic processes, time horizon, causal conditioning convention, joint distribution and finite conditional-mutual-information sum are explicit. The scope is broad within that domain but bounded by the need for the paired stochastic processes, time horizon, causal conditioning convention, joint distribution and finite conditional-mutual-information sum are explicit. The entry records a descriptive analytical identity; practical use requires the governing domain's evidence, standards, and safety obligations.
Clarity¶
The abstraction clarifies a crowded vocabulary by making the paired stochastic processes, time horizon, causal conditioning convention, joint distribution and finite conditional-mutual-information sum are explicit the center of the account. A claim should name the carrier, the governing operation or relation, the applicable assumptions, and the recognition test. A bare label is insufficient because the name Directed information can be used for a formal identity, an implementation, or a neighboring result unless carrier and convention are stated.
Manages Complexity¶
Without the abstraction, an analyst must reason directly over many local details: the carrier roles, admissibility assumptions, competing conventions, derived invariants, boundary cases, and proof or validation obligations specific to Directed information. Directed information compresses them into the roles in the structural signature. That compression permits comparison across instances without erasing the variables that determine validity. It also exposes which details may be varied safely and which are constitutive.
Abstract Reasoning¶
- Identify the carrier. State what the elements, states, objects, or observations are: the typed information theory carrier, defining objects and relations, parameters, conventions, evidence, boundary cases, and comparison targets. Reject examples whose alleged carrier belongs to a different problem. 2. Lock the constitutive rule. Express the paired stochastic processes, time horizon, causal conditioning convention, joint distribution and finite conditional-mutual-information sum are explicit independently of one notation or implementation.
Knowledge Transfer¶
Knowledge transfers strongly among subfields of information theory because they reuse the typed information theory carrier, defining objects and relations, parameters, conventions, evidence, boundary cases, and comparison targets, At each time, conditioning removes what the destination's own history already predicts, then mutual information measures the additional contribution available from the source history; summation accumulates the flow., and type the carrier, state every parameter and convention in the definition, test that the paired stochastic processes, time horizon, causal conditioning convention, joint distribution and finite conditional-mutual-information sum are explicit, compare the nearest accepted identity, and report counterexamples, uncertainty, and limiting cases.
Relationships to Other Abstractions¶
Current abstraction Directed information Domain-specific
Parents (1) — more general patterns this builds on
-
Directed information is a kind of Information Prime
The proposed strict upward parent is
prime:information.
Hierarchy path (1) — routes to 1 parentless root
- Directed information → Information → Uncertainty
Neighborhood in Abstraction Space¶
Directed information sits in a crowded region of the domain-specific corpus (14th percentile for distinctiveness): several abstractions share nearly its structure, so a description that fits it tends to fit its neighbors too.
Family — Epistemic Measurement & Causal Reasoning (19 abstractions)
Nearest neighbors
- Min-entropy — 0.93
- Information dimension — 0.93
- Asymptotic equipartition property — 0.93
- Typical set — 0.92
- Binary entropy function — 0.91
Computed from structural-signature embeddings · 2026-09-08