Compression¶
Core Idea¶
Compression is the encoding of information in a representation shorter than the original, exploiting redundancy (statistical regularity, structural predictability, perceptual unimportance)[1] to reduce the number of symbols, bits, or physical resources required to store or transmit it — either losslessly (exact reconstruction possible)[2] or lossily (controlled approximation, accepting some degradation for much greater reduction). The essential commitment is that raw information representations generally contain redundancy that can be eliminated without reducing — and often with corresponding increase in — their effective utility, and that the Shannon limit sets a hard lower bound on lossless compression (the entropy of the source) below which no lossless encoding can reach. Every compression articulation specifies (1) the source model — what is being compressed (text, image, audio, video, scientific data, code)[3] and its statistical or structural properties; (2) the loss discipline — lossless (exact reconstruction, bounded by entropy) or lossy (controlled fidelity loss, subject to distortion-rate tradeoffs); (3) the algorithm — entropy coding (Huffman, arithmetic, ANS)[4], dictionary methods (LZ77, LZ78, LZW), transform coding (DCT, wavelet), predictive coding, perceptual models; and (4) the use context — storage vs transmission, one-shot vs streaming, latency-sensitive vs bandwidth-sensitive, correctness-critical vs approximable. The construct has roots in Shannon's 1948 foundational information theory[5], Huffman's 1952 prefix code[4], Lempel-Ziv's 1977/78 dictionary methods, and the rich subsequent literature on transform and neural compression.
How would you explain it like I'm…
Making things smaller
Squishing information
Shrinking data without losing it
Structural Signature¶
For a discrete source X with symbol probabilities p(x), Shannon's source coding theorem gives the lower bound on the expected per-symbol code length of any uniquely decodable code: L ≥ H(X) = −Σ p(x) log₂ p(x), the entropy. Huffman codes are optimal (within 1 bit per symbol) for known distributions; arithmetic coding and asymmetric numeral systems (ANS) approach H(X) more closely. Dictionary methods (LZ77, LZ78) exploit repetition without explicit probability model, achieving asymptotic optimality for ergodic sources. Transform coding (JPEG, MP3) compacts energy into fewer coefficients via decorrelating transforms (DCT, wavelet), then quantizes coefficients according to perceptual weighting. Rate-distortion theory gives the lower bound on lossy compression: R(D) = min_{p(x̂|x): E[d(x,x̂)] ≤ D} I(X; X̂), so achieving distortion ≤ D requires at least R(D) bits per symbol.
What It Is Not¶
Common misclassification: Treating compression as identical to abstraction. Compression reduces representation size while (in the lossless case) preserving all information; abstraction deliberately discards or hides detail to highlight essentials. A compressed file is recoverable; an abstract model is intentionally incomplete. The two are related (both reduce cognitive or computational load) but structurally distinct — see abstraction.
Not identical to deduplication or summarization: deduplication removes identical duplicate copies (files, blocks); compression reduces redundancy within a single data stream. Summarization extracts the salient subset of content (journalistic, scientific) and is typically lossy and not recoverable in compression's structural sense.
Not unlimited: Shannon's theorem shows that lossless compression has a hard limit (the entropy). Claims of infinite compression (the "magic compression" scams) violate this limit and are mathematically impossible for generic data. Lossy compression can achieve higher ratios but at a controlled distortion cost.
Not equivalent to bigger-is-faster: compression trades computation (encoding and decoding time) for storage or bandwidth. For fast storage and slow network, compression is generally beneficial; for slow storage and fast network, or for very compute-constrained decoders, uncompressed may be faster.
Not always beneficial when nested or re-applied: compressing already- compressed data rarely helps (the first compression removes most redundancy) and often hurts (overhead of the second compression with no gain). Repeated compression of truly random or near-random data can increase its size.
Not free of correctness concerns: compression-algorithm bugs can corrupt data; compression-adaptive protocols create attack surfaces (CRIME, BREACH exploits of TLS compression); lossy compression discards information that may be important for specific downstream use cases.
Not universal in the "no free lunch" sense: no single compression algorithm is optimal for all sources. Text, images, audio, and code each benefit from different algorithms (LZ variants for text, DCT for images, predictive coding for audio). Specialized algorithms leverage source-specific structure that generic ones cannot.
Cross-references: see entropy (the information-theoretic lower bound on lossless compression); see information (the quantity being compressed); see redundancy (what compression exploits); see shannon_limit (the relevant theoretical bound); see abstraction (the related but distinct concept of structural reduction).
Broad Use¶
Compression appears in computing (file compression: zip, gzip, xz, zstd, lz4[6]; network protocols: HTTP/gzip, QUIC compression), in media (JPEG, PNG, HEIC for images[7]; MP3, AAC, Opus for audio; H.264, H.265, AV1 for video), in databases (dictionary encoding, columnar compression, delta encoding), in communications (modem codecs, cellular speech codecs), in scientific data (HDF5, NetCDF compression; genomic data formats BAM/CRAM), in machine learning (weight compression, quantization, pruning, knowledge distillation)[8], in cognitive science (chunking, schema formation in memory and learning), in biology (DNA as highly compressed phenotypic information, protein folding), in language (grammar as compression of surface forms), in journalism and education (summarization), and in any context where storage or transmission cost matters relative to the cost of encoding and decoding.
Clarity¶
Compression clarifies that information has both a surface representation and an irreducible content (entropy)[9], that redundancy in a surface representation can often be removed without loss, that the entropy sets a hard lower bound on lossless compression, that lossy compression trades fidelity for rate according to the rate-distortion curve, and that different data types have different optimal algorithms reflecting their different redundancy structures.
Manages Complexity¶
The construct manages the complexity of representing large information streams by providing algorithms parameterized by source model and distortion constraint, by tying representation size to the underlying entropy (a deep connection to probability theory)[10], and by factoring the compression problem into source modeling (what regularities to exploit) and encoding (how to encode them efficiently). The Shannon framework supports rigorous analysis and bounds.
Abstract Reasoning¶
Compression reasoning proceeds by characterizing the source (probability model, structural regularities), choosing lossless vs lossy based on use, selecting or designing an algorithm matched to the source structure[11], quantifying achievable rate (lower-bounded by entropy in the lossless case or by R(D) in the lossy case), and evaluating encoding / decoding complexity relative to the application's compute budget. It licenses design decisions (which codec for which data type, which quality level for lossy formats, whether to compress for a given use case) and theoretical analyses (how close to the Shannon bound a given algorithm gets).
Knowledge Transfer¶
| Role | Text-compression form | Image-compression form | Neural-model-compression form | Cognitive-chunking form |
|---|---|---|---|---|
| Source | Text / source code | Images / frames | Model weights | Memories / skills |
| Redundancy exploited | Character frequencies, repetitions | Spatial correlation, perceptual insignificance | Weight correlation, low effective rank | Frequency, structure, semantic grouping |
| Algorithm | Huffman, LZ77, LZ78, arithmetic, zstd | DCT (JPEG), wavelet (JPEG 2000), neural (JPEG AI) | Quantization, pruning, distillation | Chunking, schema formation |
| Lossless / lossy | Typically lossless | Typically lossy | Typically lossy | Usually lossy |
| Shannon analog | Entropy of the text source | Rate-distortion curve | Rate-accuracy tradeoff | Capacity of working memory |
An information-theory practitioner's compression reasoning transfers across text, images, audio, video, neural networks, and cognition. The structural core is redundancy identification + efficient encoding within the source's distortion budget; what varies is the substrate and the appropriate algorithm.
Examples¶
Formal/abstract¶
Huffman coding of English text: English text has highly non- uniform character frequencies ('e' ≈ 12%, 'z' ≈ 0.07%). Huffman's greedy algorithm constructs an optimal prefix code by iteratively merging the two lowest- probability symbols into a combined node, producing variable-length codes (short codes for frequent characters, long codes for rare). For English, Huffman codes achieve approximately 4.5 bits per character (compared to 8 bits for ASCII), saving nearly 45% — approaching the language entropy bound of about 4.1 bits per character. Arithmetic coding and later methods reduce further; modern compressors (zstd, brotli) combine entropy coding with dictionary methods to exploit repetition at multiple scales. This classical example illustrates both the theoretical framework (Shannon bound, optimality of Huffman) and the practical machinery (per-symbol encoding tables, prefix-free codes).
Mapped back: Huffman coding exemplifies the structural signature of the entropy as theoretical lower bound on bits per symbol and the encoder-decoder pair preserving information — variable-length prefix codes implement the redundancy-extraction principle that compression formalizes; the source-distribution as redundancy structure manifests as character frequencies, and the lossless-versus-lossy distinction collapses to the bound-vs-floor framing.
Applied/industry¶
Cognitive chunking of a phone number: A 10-digit phone number like 4159876543 is at the edge of working-memory capacity (Miller's 7±2). Most people mentally chunk it: 415-987-6543 (area code— exchange—last four), representing the same information with fewer chunks. Further meaningful chunking (recognizing 415 as San Francisco, 987 as a repeated-pattern) further reduces effective cognitive load. This is compression of information into forms that fit cognitive capacity constraints — analogous to source coding but with perceptually / semantically significant structure rather than statistical redundancy. The structural match is real: reducing the symbol count needed to represent the same information by exploiting structure (area-code groupings, local-exchange significance), though the "algorithm" is learned cognitive schema rather than a formal encoder.
Mapped back: Cognitive chunking of phone numbers exemplifies the same compression abstraction in human memory — chunking exploits the source distribution (familiar number patterns), achieves entropy reduction (fewer working-memory chunks for the same information), and trades off lossy abstraction (losing exact-digit ordering when chunks become semantic) for tractability; the encoder-decoder pair becomes the chunking-and-recall procedure.
Structural Tensions¶
T1 — Name: Shannon bound limits lossless compression irreducibly. No lossless compression of arbitrary data can systematically exceed the source's entropy, and this bound is mathematically proven. "Universal" compressors like Lempel-Ziv can approach the bound asymptotically but cannot beat it, and claims of magic compression violate this limit. Engineering analyses that assume arbitrarily tight compression systematically mis-estimate storage and bandwidth requirements.
T2 — Name: Lossy compression trades fidelity for rate; reuse for unanticipated purposes risks bias. Lossy codecs (JPEG, MP3, HEVC) are designed for specific perceptual loss criteria (visual artifacts, frequency masking), and discarded information may matter for non-perceptual reuse (scientific image analysis, machine learning). Lossy-compressed data reused in analytical or downstream contexts can introduce subtle biases or artifacts in results, and these distortions are often unacknowledged by downstream users.
T3 — Name: Compression-based side channels leak information via length and timing. Adaptive compression of sensitive data mixed with attacker-controllable content creates side-channel vulnerabilities: the CRIME and BREACH attacks on TLS exploited compression-length correlations to infer secrets, and compression-timing information can leak across security boundaries. Compression applied to heterogeneous (mixed-sensitivity) data without explicit information-leakage modeling creates unintended disclosure channels.
T4 — Name: Compression imposes asymmetric computational costs; context determines benefit. Encoding is typically much more expensive than decoding (often 10-100x more), which is acceptable for one-to-many broadcast (video delivery) but problematic for constrained peer-to-peer or IoT scenarios. Compression applied at scale without accounting for encoding cost (e.g., logging infrastructure, request processing pipelines) can cause CPU bottlenecks, and decompression latency can block interactive applications requiring sub-millisecond response times.
T5 — Name: Universal compression fails; algorithm must match source structure. No single algorithm is optimal for all data: text benefits from Lempel-Ziv dictionaries, images from transform coding (DCT, wavelet), audio from predictive coding[1], and video from motion compensation plus transform coding. Specialized algorithms leverage source-specific redundancy structure that generic compressors cannot, so choosing compression requires understanding the data.
T6 — Name: Compressed-data formats ossify; algorithm evolution breaks backward compatibility. Once a compression format is standardized and widely deployed (JPEG in photographs, MP3 in audio), format changes are nearly irreversible due to vast installed bases and ecosystem dependencies. Newer, more-efficient formats (HEIC, WebP, Opus) exist but displace slowly because decompression support must reach end-user devices and decoders must be portable across platforms[12].
Structural–Framed Character¶
Compression sits at the structural end of the structural–framed spectrum: it is a pure relational pattern, the same in any domain where it appears, and nothing about its meaning depends on a particular field's vocabulary or assumptions. It names the encoding of information in a shorter representation by exploiting redundancy—statistical regularity, structural predictability, or perceptual unimportance—whether losslessly or with controlled approximation.
The core idea is anchored by a formal result, Shannon's source-coding theorem, which fixes how short any uniquely decodable code can be; that bound and the redundancy-exploiting logic transfer without alteration from text and image files to audio streams and data archives. The notion carries no built-in approval or disapproval—more compression is not virtuous, only more efficient. Its origin is mathematical rather than institutional, it can be stated without reference to human practices, and applying it feels like recognizing redundancy that is already in the source. On every diagnostic, it reads structural.
Substrate Independence¶
Compression is a highly substrate-independent prime — composite 4 / 5 on the substrate-independence scale. Its signature — exploiting statistical regularity through encoding to produce a shorter representation — is substrate-agnostic and instantiates concretely across information theory (Huffman coding), psychology (cognitive chunking), linguistics (morphology), and social systems (hierarchies that reduce coordination complexity). The transfer is explicit and the examples genuinely span multiple substrates. What keeps it a notch below the ceiling is the gravitational pull of its information-theoretic home, which frames much of how the prime is understood even as it travels.
- Composite substrate independence — 4 / 5
- Domain breadth — 4 / 5
- Structural abstraction — 4 / 5
- Transfer evidence — 4 / 5
Relationships to Other Primes¶
Parents (3) — more general patterns this builds on
-
Compression is a kind of Abstraction
Compression is a specialization of abstraction in which the purpose-relative retention is reduction of representational length: keep the information-theoretic content needed for exact or approximate reconstruction, drop the statistical or perceptual redundancy. It inherits abstraction's general commitment to purpose-relative retention of structure with explicit naming of what is kept and dropped, and specializes by fixing the purpose to storage or transmission economy and the projection to one that shortens symbol count, with the Shannon limit setting the floor on lossless reduction.
-
Compression is a kind of Aggregation
Compression encodes information in a shorter representation by exploiting redundancy, deliberately losing or restructuring detail to retain the features that matter for reconstruction or downstream use. That is the move of Aggregation: collapsing many items into a unified form that keeps chosen features while suppressing granular detail. Compression specializes aggregation by tying the suppressed detail to redundancy or perceptual unimportance and by holding a reconstruction or fidelity criterion as the design constraint.
-
Compression is a kind of Optimization
Compression seeks the shortest encoding of a source under a chosen fidelity criterion — exact reconstruction for lossless, bounded distortion for lossy — and trades coding cost against quality at the chosen operating point. That is the optimization triplet: decision variable (the encoding), objective (representation length), and constraint (fidelity). Compression specializes optimization to the encoding-length objective, with rate-distortion theory and entropy bounds setting the achievable frontier.
Children (3) — more specific cases that build on this
-
Chunking is a kind of Compression
Chunking is a specialization of compression in which the redundancy being exploited is structural relatedness among items, and the encoding shrinks the count of units working memory must track by binding them into one meaningful chunk. It inherits the general compression commitment that representational length can be reduced when the source contains predictable or relational structure, and specializes by locating the encoding in cognitive working memory: capacity is measured in chunks rather than raw elements, so restructuring raises effective capacity without enlarging the store.
-
Dimensionality Reduction is a kind of Compression
Dimensionality reduction is a specialization of compression in which the redundancy being exploited is dimensional: high-dimensional data lies on or near a low-dimensional manifold, and a transformation projects it onto that lower-dimensional representation while preserving variance, distances, or predictive information. It inherits the general compression commitment that redundant structure can be eliminated without losing what matters and that the choice between lossless and lossy reduction is governed by which features must be preserved. The specialization fixes the redundancy to dimensional correlations rather than symbol-level statistics.
-
Predictive Coding presupposes Compression
Predictive coding presupposes compression because its central move is to suppress the expected component of an incoming signal and transmit only the residual prediction error: this is exactly the compression strategy of exploiting predictable structure to shorten what must be encoded. Compression supplies the general principle that redundant statistical regularity can be removed without loss; predictive coding instantiates it by making a generative model the source of the predicted regularity, so only the surprising remainder propagates as both message and learning signal.
Path to root: Compression → Aggregation
Neighborhood in Abstraction Space¶
Compression sits in a sparse region of abstraction space (96th percentile for distinctiveness): few abstractions share its structure, so a faithful description tends to retrieve it precisely rather than landing on a neighbor.
Family — Computational Process & Control (12 primes)
Nearest neighbors
- Redundancy — 0.77
- Algorithm — 0.75
- Complexity (Time/Space) — 0.73
- Predictive Coding — 0.72
- Multiplexing — 0.72
Computed from structural-signature embeddings · 2026-05-29
Not to Be Confused With¶
Compression must be distinguished from Dimensionality Reduction, its closest neighbor (similarity 0.688). Both reduce data size, but they operate on opposite principles. Compression reduces representation size by exploiting redundancy within the data while preserving information content (lossless compression) or achieving controlled information loss within a fidelity budget (lossy compression); the information reduction (if any) is bounded and measured. Dimensionality Reduction deliberately discards low-variance or low-information dimensions to make high-dimensional data tractable for visualization, analysis, or learning; information loss is intentional and often substantial. A text file compressed by lossless Huffman coding retains all information; a dataset reduced from 1000 dimensions to 2 dimensions for visualization loses most information. Compression preserves Shannon entropy (or bounded divergence from it); dimensionality reduction discards dimensions. Compression is about encoding economy; dimensionality reduction is about problem tractability. A 1000-dimensional dataset can be losslessly compressed if it has redundancy; it is reduced to 2 dimensions to enable visualization at the cost of information loss.
Compression is also distinct from Chunking, though both reduce the number of units a system must track. Chunking is a cognitive process that groups items into larger meaningful units, reducing the number of working-memory chunks needed to hold information—a phone number 4159876543 becomes 415-987-6543, three chunks instead of ten. Compression is an information-theoretic encoding that reduces the bit-length of a representation by exploiting statistical regularities—the same phone number can be compressed to fewer bits by entropy coding if its digits follow a non-uniform distribution. Chunking operates on semantic meaningfulness (recognizing that 415 is an area code reduces chunks); compression operates on statistical structure (using fewer bits to encode frequent digits). Chunking is cognitive and domain-specific; compression is formal and substrate-agnostic. Both reduce the number of units, but in different substrates and mechanisms.
Nor is compression identical to Representation. Representation is the faithful mapping of a target system onto a medium, preserving selected structural properties so that the representation can stand in for the target for specific purposes—a map represents territory by preserving spatial relationships; a model represents a physical system by preserving causal dynamics. Compression is the reduction of representation size by exploiting redundancy, often accepting information loss (lossy case) to achieve shorter encoding. A high-fidelity representation of a photograph is detailed; a compressed JPEG is smaller and lossy. Representation prioritizes fidelity to structure; compression prioritizes encoding economy. A good representation can be expanded (all information is there to be extracted); a lossy-compressed file is permanently degraded.
Finally, compression is distinct from Entropy (Thermodynamic Sense). Thermodynamic entropy quantifies the number of accessible microstates consistent with a macroscopic state—a measure of disorder, possibility, or information potential in a physical system. Information-theoretic entropy (Shannon entropy) quantifies the average information content of a probability distribution, setting a lower bound on lossless compression rate. Compression exploits information-theoretic entropy by using statistical regularities in data to reduce encoding length; it has no direct relationship to thermodynamic entropy except through the metaphorical connection. A string with high Shannon entropy is hard to compress; thermodynamic entropy is a different concept about physical systems. Information theory borrowed the term "entropy" from thermodynamics due to mathematical similarities, but they measure different phenomena.
Solution Archetypes¶
Solution archetypes in the catalog that build on this prime — directly (this prime is a source ingredient) or as a related prime.
Built directly on this prime (6)
- Accumulation Compaction
- Chunked Information Design
- Cognitive Load Reduction
- Dimensionality Reduction for Signal
- Index-Based Retrieval
- Task-Relevant Compression
Also a related prime in 21 archetypes
- Aggregation Bias Detection and Correction
- Aggregation Function Design and Weighting
- Archetype Pattern Indexing
- Cascaded Hierarchical Recognition
- Coarse-Graining
- Cognitive Representation Externalization
- Degrees-of-Freedom Reduction
- Emergent Formalization
- Equivalence Class Consolidation
- Equivalence-Relation Refinement and Coarsening
Notes¶
Additional canonical reference: [4].
Held at High confidence. Information-theory foundational construct with wide applied substrates. Entry emphasizes the Shannon-entropy bound, distinguishes lossless from lossy compression, distinguishes compression from abstraction, and catalogs the security / attack-surface failure mode. Cross-DP-25 notes: compression quantifies information content via entropy and algorithmic complexity; reproducibility_replicability is the meta-validation framework; missing_data_mechanisms_mcar_mar_mnar identifies structural threats to complete-data assumptions.
References¶
[1] MacKay, D. J. C. (2003). Information Theory, Inference, and Learning Algorithms. Cambridge: Cambridge University Press. http://www.inference.org.uk/mackay/itila/. MacKay integration of information theory with machine learning. ↩
[2] Cover, T. M., & Thomas, J. A. (2006). Elements of Information Theory (2nd ed.). Wiley-Interscience. Standard information-theory text: separates channel noise (an apparatus-and-environment property limiting capacity) from intrinsic source entropy (a property of the underlying signal), clarifying that noise is technological while source structure is fundamental. ↩
[3] Rissanen, J. (1978). Modeling by shortest data description. Automatica, 14(5), 465–471. https://doi.org/10.1016/0005-1098(78)90005-5. Rissanen Minimum Description Length principle. ↩
[4] Huffman, D. A. (1952). A method for the construction of minimum-redundancy codes. Proceedings of the Institute of Radio Engineers, 40(9), 1098–1101. https://doi.org/10.1109/JRPROC.1952.273898. Huffman optimal prefix-code algorithm. ↩
[5] Shannon, C. E. (1948). "A mathematical theory of communication." The Bell System Technical Journal, 27(3), 379–423. ↩
[6] Lempel, A., & Ziv, J. (1977). A universal algorithm for sequential data compression. IEEE Transactions on Information Theory, 23(3), 337–343. https://doi.org/10.1109/TIT.1977.1055714. Lempel-Ziv universal dictionary-compression algorithm. ↩
[7] Blahut, R. E. (1972). Computation of channel capacity and rate-distortion functions. IEEE Transactions on Information Theory, 18(4), 460–473. https://doi.org/10.1109/TIT.1972.1054855. Rate-distortion theory formalizing lossy-compression trade-offs. ↩
[8] Li, M., & Vitányi, P. M. B. (2008). An Introduction to Kolmogorov Complexity and Its Applications (3rd ed.). New York: Springer. https://doi.org/10.1007/978-0-387-49820-1. Li-Vitanyi Kolmogorov-complexity foundations. ↩
[9] Kolmogorov, A. N. (1965). "Three approaches to the quantitative definition of information." Problems of Information Transmission, 1(1), 1–7. (Originating treatment of Kolmogorov complexity / algorithmic information theory; defines incompressibility-based randomness for individual sequences. Parallel independent work: Solomonoff 1964, Chaitin 1969.) ↩
[10] Solomonoff, R. J. (1964). "A formal theory of inductive inference." Information and Control, 7(1), 1–22. (Originating treatment of algorithmic probability and universal inductive inference; establishes theoretical foundations for learning from data; parallel independent work to Kolmogorov and Chaitin.) ↩
[11] Lempel, A., & Ziv, J. (1978). Compression of individual sequences via variable-rate coding. IEEE Transactions on Information Theory, 24(5), 530–536. https://doi.org/10.1109/TIT.1978.1055934. Lempel-Ziv universal compression asymptotic optimality. ↩
[12] Salus, P. H. (2012). The Daemon, the GNU and the Penguin. Groklaw. Format standardization and backward-compatibility constraints. ↩
[13] Hamming, R. W. (1950). "Error detecting and error correcting codes." The Bell System Technical Journal, 29(2), 147–160.
[14] Rivest, R. L., Shamir, A., & Adleman, L. (1978). "A method for obtaining digital signatures and public-key cryptosystems." Communications of the ACM, 21(2), 120–126.
[15] Pacioli, L. (1494). Summa de arithmetica, geometria, proportioni et proportionalita [Summary of Arithmetic, Geometry, Proportions and Proportionality]. Paganinus de Paganinis.
[16] Bonwick, J., Ahrens, M., Henson, V., Maybee, M., & Shellenbaum, M. (2005). "ZFS: The Last Word in Filesystems." Whitepaper.
[17] Codd, E. F. (1970). "A relational model of data for large shared data banks." Communications of the ACM, 13(6), 377–387.
[18] Merkle, R. C. (1987). "A digital signature based on a conventional encryption function." In Advances in Cryptology — CRYPTO '87.
[19] National Institute of Standards and Technology. (2015). "SHA-3 Standard: Permutation-Based Hash and Extendable-Output Functions." NIST FIPS 202.
[20] Shannon, C. E. (1949). Communication in the presence of noise. Proceedings of the IRE, 37(1), 10–21. Foundational sampling theorem: bandlimited signals are uniquely determined by samples taken at the Nyquist rate; below this rate, undersampling produces false frequency components.
[21] Chaitin, G. J. (1969). "On the length of programs for computing finite binary sequences." Journal of the ACM, 16(1), 145–159. (Originating treatment of Chaitin's omega and algorithmic randomness; parallel independent work to Kolmogorov and Solomonoff.)
[22] Kelsey, J., Schneier, B., & Wagner, D. (1997). Mod n cryptanalysis, with applications against RC5P and M6. In Fast Software Encryption (pp. 139–155). Berlin: Springer. Compression side-channel attacks (CRIME, BREACH) on security protocols.