Lecture 3: Entropy, Relative Entropy, and Mutual Information¶
Weissman, T., Devorah, A., Hallac, D., & Shutzberg, K. (2015). Lecture 3: Entropy, Relative Entropy, and Mutual Information: Entropy, Relative Entropy, and Mutual Information. Stanford EE376A.
Cited by¶
1 citation across 1 artifact.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Subadditivity
- Shannon introduced the governing entropy quantities, and Stanford's information-theory notes present the inequality, equality condition, and mutual-information identity together.
This sourceStates \(H(X,Y)\leq H(X)+H(Y)\), equality under independence, and the mutual-information slack identity.
- Shannon introduced the governing entropy quantities, and Stanford's information-theory notes present the inequality, equality condition, and mutual-information identity together.
Verification¶
This reference passed the adversarial substantiation pipeline: it was checked to exist and to support the claim it is attached to. See how references were verified.
Registry ID ref:cbff7bade79d · see in the full table