A Mathematical Theory of Communication¶
Shannon, C. E. (1948). A Mathematical Theory of Communication. Bell System Technical Journal, 27(3), 379-423.
Cited by¶
32 citations across 31 artifacts.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Aggregation
- Aggregation names the moment when multiple distinct entities are deliberately collapsed into a unified measure or category—a designed moment of information loss whose generality is captured by Shannon's (1948) information-theoretic framing of the channel between source and summary.
This sourceInformation-theoretic framing of the source-to-summary channel and compression; supports naming the deliberate information loss that defines aggregation.
Supported in partVerified against the work's full text
Supplies the general source-channel-recipient framework, including transmission within a fidelity tolerance, i.e. accepted loss. It never discusses aggregation or summaries.
“Practically, we are not interested in exact transmission when we have a continuous source, but only in transmission to within a certain tolerance.”
- The first structural tension—the irreversibility of aggregation as an operation that destroys information—follows directly from Shannon's (1948) data-processing inequality: no post-hoc transformation of the summary y can recover information about the inputs x₁, …, xₙ that was discarded in forming y.
This sourceEstablishes the data-processing inequality; supports the irreversibility tension — no transformation of a summary can recover the inputs' discarded information.
- Aggregation names the moment when multiple distinct entities are deliberately collapsed into a unified measure or category—a designed moment of information loss whose generality is captured by Shannon's (1948) information-theoretic framing of the channel between source and summary.
- Amplification
- Channel
- There is (1) a two-endpoint coupling (source to receiver) with definite directionality; (2) a capacity — finite, usually quantifiable — that bounds how much can cross per unit time; (3) an alphabet or codebook — the channel's admissible input set, often more constrained than what the source would prefer; (4) a noise or distortion profile — the channel transforms the input probabilistically, making the receiver's reconstruction non-trivial; and (5) a medium — the substrate through which transmission occurs, whose physics determines the other four.
This sourceFounding paper of information theory; defines the channel, channel capacity, the noisy-channel coding theorem, and the binary symmetric channel.
Supported in partVerified against the work's full text
Shannon supplies the channel as a medium between transmitter and receiver, a definite capacity, and probabilistic noise; the five-part synthesis and "medium determines the other four" are the encyclopedia's own.
“3. The channel is merely the medium used to transmit the signal from transmitter to receiver.”
- There is (1) a two-endpoint coupling (source to receiver) with definite directionality; (2) a capacity — finite, usually quantifiable — that bounds how much can cross per unit time; (3) an alphabet or codebook — the channel's admissible input set, often more constrained than what the source would prefer; (4) a noise or distortion profile — the channel transforms the input probabilistically, making the receiver's reconstruction non-trivial; and (5) a medium — the substrate through which transmission occurs, whose physics determines the other four.
- Channel Capacity
- Fourth, a capacity: a maximum reliable rate, increasing with bandwidth and with the logarithm of the signal-to-noise ratio, that no encoding can exceed.
This sourceDefines channel capacity as the maximum mutual information over input distributions, and proves the achievability-below / impossibility-above coding theorem.
SupportedVerified against the work's full text
Shannon's Theorem 17 gives the capacity of a band-W channel in white noise as W log((P+N)/N) and says no encoding system can exceed that rate without a definite positive error frequency.
“Theorem 17: The capacity of a channel of band W perturbed by white thermal noise power N when the average transmitter power is limited to P is given by ... It is not possible to transmit at a higher rate by any encoding system without a definite positive frequency of errors.”
- Fourth, a capacity: a maximum reliable rate, increasing with bandwidth and with the logarithm of the signal-to-noise ratio, that no encoding can exceed.
- Compression
- The construct has roots in Shannon's 1948 foundational information theory
This sourceFounds information theory; introduces entropy H(X), redundancy, and the source-coding theorem fixing the entropy lower bound on lossless representation. SUPPORTS marker 171 (Shannon's 1948 foundational information theory) and the Shannon-limit framing throughout. DOI (Wiley) verified; also archived at archive.org/details/bstj27-3-379.
SupportedVerified against the work's full text
Shannon's 1948 paper defines source entropy, relative entropy as the maximum compression possible, and redundancy, laying the theoretical base for compression.
“This is the maximum compression possible when we encode into the same alphabet. One minus the relative entropy is the redundancy.”
- The construct has roots in Shannon's 1948 foundational information theory
- Conditional Entropy
- Conditional Probability
- reinforcement-learning policies are conditional distributions over actions given states. Game theory and information economics. A player's belief about others' types after a signal is a conditional distribution; sequential equilibria, Bayesian games, and signalling models all rest on conditioning and the discipline of updating it. Forecasting and engineering. Forecasts are conditional on currently available data, with the conditional reduction in uncertainty being the forecast's information content; fault-tree analysis is structured by \(P(\text{failure}\mid\text{component failed})\). Communication theory. Mutual information \(I(X;Y) = H(X) - H(X\mid Y)\) is a comparison of unconditional and conditional uncertainties
This sourceFounds information theory; mutual information I(X;Y) = H(X) − H(X|Y) is built from conditional distributions as a comparison of unconditional and conditional uncertainty.
Supported in partVerified against the work's full text
Shannon's rate R = H(x) minus equivocation is the unconditional-minus-conditional uncertainty form of mutual information, supporting only the communication-theory item.
“seems reasonable to use the conditional entropy of the message, knowing the received signal, as a measure of this missing information.”
- reinforcement-learning policies are conditional distributions over actions given states. Game theory and information economics. A player's belief about others' types after a signal is a conditional distribution; sequential equilibria, Bayesian games, and signalling models all rest on conditioning and the discipline of updating it. Forecasting and engineering. Forecasts are conditional on currently available data, with the conditional reduction in uncertainty being the forecast's information content; fault-tree analysis is structured by \(P(\text{failure}\mid\text{component failed})\). Communication theory. Mutual information \(I(X;Y) = H(X) - H(X\mid Y)\) is a comparison of unconditional and conditional uncertainties
- Data Integrity
- The essential commitment is that data without explicit integrity protection is progressively corrupted by bit rot, transmission errors, software bugs, operator mistakes, and adversarial manipulation; that detecting corruption requires redundancy or cryptographic verification; and that different threats require different mechanisms
This sourceFounds information theory; establishes redundancy and channel coding as the basis for reliable transmission over noisy channels. Anchors D30-106, D30-113, D30-114 as the information-theoretic foundation for detecting corruption via redundancy. Verified. (See flag on D30-106: Shannon grounds redundancy-for-detection but not the specific 'different threats require different mechanisms' phrasing.)
Supported in partVerified against the work's full text
Shannon shows redundancy lets errors in a noisy channel be reduced or corrected, and that it must suit the noise structure; he does not treat bit rot, bugs, operator error or adversaries.
“The redundancy must be introduced in the proper way to combat the particular noise structure involved.”
- The essential commitment is that data without explicit integrity protection is progressively corrupted by bit rot, transmission errors, software bugs, operator mistakes, and adversarial manipulation; that detecting corruption requires redundancy or cryptographic verification; and that different threats require different mechanisms
- Discrete vs. Continuous (Quantization)
- Classical examples: position in Newtonian mechanics, temperature, voltage (pre-quantization), real-valued economic indicators.
This sourceFounds information theory; formally treats discrete vs continuous information sources/channels and states the sampling theorem (Theorem 13) — a band-limited signal is fully determined by samples at twice its highest frequency.
Supported in partVerified against the work's full text
Shannon treats the continuous case as the limit of discrete subdivisions, but says nothing about hbar, lattice QFT or an LSB quantization step.
“To a considerable extent the continuous case can be obtained through a limiting process from the discrete case by dividing the continuum of messages and signals into a large but finite number of small regions”
- Classical examples: position in Newtonian mechanics, temperature, voltage (pre-quantization), real-valued economic indicators.
- Encoding And Decoding
- In information theory, the Shannon origin, messages are encoded into channel symbols, transmitted, and decoded back, and the entire apparatus of error-correcting codes, source coding, and channel coding lives inside this pair.
This sourceFounds the source–encoder–channel–decoder framework and the redundancy-versus-recoverability trade governing source and channel coding.
Supported in partVerified against the work's full text
Shannon sets up the source-transmitter-channel-receiver chain with encoding and decoding as its operations, but does not say error-correcting, source and channel coding all live inside this pair.
“We have yet to represent mathematically the operations performed by the transmitter and receiver in en- coding and decoding the information.”
- In information theory, the Shannon origin, messages are encoded into channel symbols, transmitted, and decoded back, and the entire apparatus of error-correcting codes, source coding, and channel coding lives inside this pair.
- Entropy (Thermodynamic Sense)
- Information-theoretic entropy H = −Σ p_i log p_i shares mathematical form with Gibbs entropy
This sourceIntroduces information entropy H = −Σ p_i log p_i as a measure of uncertainty; the functional form shared with Gibbs entropy.
SupportedVerified against the work's full text
Shannon gives H = -K Σ p_i log p_i and says its form is that of entropy in statistical mechanics, with p_i the probability of a phase-space cell.
“The form ofH will be recognized as that of entropy as defined in certain formulations of statistical mechanics8 where pi is the probability of a system being in cell i of its phase space.”
- Information-theoretic entropy H = −Σ p_i log p_i shares mathematical form with Gibbs entropy
- Logarithmic Perception and Encoding
- Engineering displays — Bode plots, log-log plots for power laws, and semi-log axes wherever data span many decades. Information theory — bits are log of the alphabet; entropy is built on log-probability; the log base reflects the measurement scale.
This sourceDefines information and entropy on the logarithm of probability, with the log base setting the measurement unit (bits).
Supported in partVerified against the work's full text
Shannon supports the information-theory half: logarithmic information measure, base 2 giving bits, entropy as a sum of p log p; it says nothing on Bode plots or log-log/semi-log engineering displays.
“The choice of a logarithmic base corresponds to the choice of a unit for measuring information. If the base 2 is used the resulting units may be called binary digits, or more briefly bits”
- Engineering displays — Bode plots, log-log plots for power laws, and semi-log axes wherever data span many decades. Information theory — bits are log of the alphabet; entropy is built on log-probability; the log base reflects the measurement scale.
- Measurement Uncertainty and Observational Noise
- Multiplexing
- It separates two layers (the logical channels the users see and the single physical channel that actually carries them) and names the reversible combining operation that bridges them.
Supported in partVerified against the work's full text
Shannon shows that no encoding can push the information rate above the channel capacity, which underlies the capacity bound; he does not discuss sharing a channel among many users.
“Thus no matter how we encode the binary digits to obtain the signal, or how we decode the received signal to recover the message, the discrete rate for the binary digits does not exceed the channel capacity”
- It separates two layers (the logical channels the users see and the single physical channel that actually carries them) and names the reversible combining operation that bridges them.
- Perturbation
- Realized vs Possible Outcomes
- Software testing — full execution paths are the possibility set; paths exercised by the suite are the realized set; coverage reports on the gap. Information theory — channel capacity is the possibility set; the rate achieved is the realized; the gap is engineering inefficiency.
This sourceDefines channel capacity as the possibility ceiling against which an achieved transmission rate is measured.
Supported in partVerified against the work's full text
Shannon defines capacity as the maximum possible rate and the ratio of actual rate to capacity as coding efficiency, supporting the information-theory half but not the software-testing half.
“The ratio of the actual rate of transmission to the capacity C may be called the efficiency of the coding system.”
- Software testing — full execution paths are the possibility set; paths exercised by the suite are the realized set; coverage reports on the gap. Information theory — channel capacity is the possibility set; the rate achieved is the realized; the gap is engineering inefficiency.
- Redundancy
- Listed in the references but not attached to a specific claim.
- Selective Propagation
- Once they are separated the channel becomes an object in its own right, with parameters to be measured rather than a transparency to be assumed, and a disagreement between a near measurement and a far one stops being a discrepancy to be resolved and becomes a datum about the path between them.
Supported in partVerified against the work's full text
Treats source and channel as separate objects, the channel described by its own transition probabilities and capacity; the near-versus-far disagreement-as-datum reading is not in Shannon.
“If a noisy channel is fed by a source there are two statistical processes at work: the source and the noise.”
- Once they are separated the channel becomes an object in its own right, with parameters to be measured rather than a transparency to be assumed, and a disagreement between a near measurement and a far one stops being a discrepancy to be resolved and becomes a datum about the path between them.
- Signal Decay and Fadeout
- Alternatively, SNR may degrade during propagation if the signal weakens faster than noise (or if noise grows independently), in which case SNR and signal decay are correlated but not identical, a separation Shannon (1948) makes precise in his foundational information-theoretic framework, where channel capacity depends on the SNR ratio rather than on absolute signal magnitude.
Supported in partVerified against the work's full text
Shannon gives channel capacity as a function of signal power P and noise power N (the P/N relationship), but does not discuss SNR degrading during propagation or a signal-decay/SNR distinction.
“Theorem 17: The capacity of a channel of band W perturbed by white thermal noise power N when the average transmitter power is limited to P is given by”
- Alternatively, SNR may degrade during propagation if the signal weakens faster than noise (or if noise grows independently), in which case SNR and signal decay are correlated but not identical, a separation Shannon (1948) makes precise in his foundational information-theoretic framework, where channel capacity depends on the SNR ratio rather than on absolute signal magnitude.
- Subadditivity
- Shannon introduced the governing entropy quantities, and Stanford's information-theory notes present the inequality, equality condition, and mutual-information identity together.
This sourceFoundational primary source for entropy and joint/conditional information quantities.
- Shannon introduced the governing entropy quantities, and Stanford's information-theory notes present the inequality, equality condition, and mutual-information identity together.
- Vantage-Induced Omission
- Yield Loss
- This connects to a family of impossibility results across fields: no real engine reaches Carnot efficiency, no real code reaches channel capacity, no real market reaches Pareto optimality, no real controller eliminates error.
This sourceEstablishes channel capacity as the maximum reliable transmission rate — the information-theoretic limit no real code can exceed, one member of the family of physical-limit results the claim groups.
Supported in partVerified against the work's full text
Shannon proves no code can transmit reliably above channel capacity C, one member of the limit family; he does not say real codes never reach C, nor cover Carnot, Pareto or controllers.
“It is possible to send information at the rate C through the channel with as small a frequency of errors or equivocation as desired by proper encoding.”
- This connects to a family of impossibility results across fields: no real engine reaches Carnot efficiency, no real code reaches channel capacity, no real market reaches Pareto optimality, no real controller eliminates error.
Domain-specific¶
- Binary entropy function
- Bit Rate
- Digital Data
- Error-Correcting Code
- Listed in the references but not attached to a specific claim.
- Information Entropy
- Shannon derived this functional from requirements for a measure of choice and used it to analyze discrete communication sources.
This sourceThe original states the discrete entropy functional and its properties, distinguishes finite-state source entropy per symbol, and proves the qualified noiseless-channel theorem.
- Shannon derived this functional from requirements for a measure of choice and used it to analyze discrete communication sources.
- Shannon–Weaver Communication Model
- Units of Information
- Variable-Length Encoding
Mechanisms¶
- Error Correction
- The theory even sets the ceiling: for a given noise level there is a hard limit on how much reliable throughput any code can achieve, so the trade is always redundancy against rate.
This sourceDefines channel capacity as the maximum information rate achievable with arbitrarily small error and rules out reliable transmission above it.
- The theory even sets the ceiling: for a given noise level there is a hard limit on how much reliable throughput any code can achieve, so the trade is always redundancy against rate.
Verification¶
Does it exist? Confirmed. This work's DOI resolves to a registered record, which fixes its identity. That is all it fixes.
Does it back the claim? Read against the text for 15 of 32 citations: 3 supported, 12 supported in part. Each verdict is shown under its citation below, with what in the work backs the sentence.
Was it audited? Yes. A second, independent pass read the citation against the article text and recorded a verdict.
Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.
See how references were verified.
Links previously used in the corpus¶
Before the registry existed this work was also linked 5 other ways.
- https://doi.org/10.1002/j.1538-7305.1948.tb01338.x ×19
- https://doi.org/10.1002/j.1538-7305.1948.tb00917.x ×3
- https://onlinelibrary.wiley.com/doi/10.1002/j.1538-7305.1948.tb01338.x ×1
- https://www.cs.yale.edu/homes/yry/readings/general/shannon1948.pdf ×1
- https://www.press.uillinois.edu/books/?id=p725487 ×1
Registry ID ref:99a36c426d12 · see in the full table