Information Theory

← Back to Domain-Specific Abstractions by Origin Domain

94 domain-specific abstractions whose origin domain is Information Theory. They span 63 subdomains — sort by that column to group them, or click any subdomain to filter to it.

Abstraction	Subdomain	Description
Adaptive differential pulse-code modulation	Digital Signal Processing	A predictive audio coding method that quantizes sample differences while adapting the quantizer scale to recent signal behavior.
Adaptive feedback cancellation	Signal ProcessingAcoustic Feedback Control	An electro-acoustic control method that continually estimates a time-varying feedback path and subtracts its predicted contribution to suppress howling while preserving the desired signal.
Algebraic geometry code	Coding Theory	An error-correcting linear code obtained by evaluating functions or taking residues on rational points of an algebraic curve over a finite field.
Asymptotic equipartition property	—	The information-theoretic property that long source sequences concentrate on a typical set whose members have nearly equal exponential probability.
Bennett's Laws	Quantum Information TheoryQuantum Communication Resource Accounting	A four-relation resource preorder for ideal two-party quantum communication: a transmitted qubit can simulate a classical bit channel or distribute one ebit, while an ebit plus one transmitted qubit simulates two classical bits and an ebit plus two classical bits simulates one transmitted qubit.
Bilinear Transform	Signal ProcessingAnalog to Digital System Transformation	A fractional-linear substitution mapping the continuous-time complex frequency plane to the discrete-time z-plane, carrying the stable left half-plane inside the unit circle while warping frequencies.
Binary entropy function	—	The Shannon entropy of a Bernoulli variable as a concave function of its success probability.
Binary erasure channel	—	A memoryless channel model that delivers each input bit correctly or replaces it with an explicit erasure symbol at a fixed probability.
Blahut–Arimoto algorithm	Iterative Optimization	A family of alternating iterative optimization algorithms for channel capacity and rate-distortion problems that updates distributions until the information-theoretic objective converges.
Bruun's FFT algorithm	Signal ProcessingFast Fourier Transforms	A fast Fourier transform based on recursive real-coefficient factorization of the transform polynomial, postponing complex arithmetic until a final reconstruction stage.
Chirplet transform	Signal Processing	A time-frequency transform correlating a signal with localized chirps whose frequency changes within each analysis atom.
Circular convolution	Signal ProcessingDiscrete Convolution	A convolution on periodic sequences in which indices wrap modulo a fixed length, corresponding to pointwise multiplication under the discrete Fourier transform.
Colors of noise	Signal Processing	A classification of stochastic signals by the frequency dependence of their power spectral density.
Constant-Q transform	Signal Processing	A time–frequency transform whose analysis bins maintain a constant ratio of center frequency to bandwidth, producing logarithmic frequency resolution.
Constant-weight code	Coding Theory	A binary codebook whose equal-length words all contain the same fixed number of 1s, with error behavior depending on the selected words.
Convolutional code	Coding Theory	A convolutional code maps a stream of input symbols to redundant output symbols through a finite-memory convolutional encoder, so each output depends on the current input and a bounded history and can be decoded from trellis structure.
Cross-entropy	—	The expected negative log probability that a model distribution q assigns to outcomes actually drawn from a distribution p.
Dependent component analysis	Signal Processing	A blind-source-separation method recovering mutually independent groups of components while allowing dependence among members of each group.
Directed information	—	An information-theoretic quantity summing the conditional information that the present and past of one process provide about each current value of another given the latter's past.
Discrete Fourier transform	Signal Processing	An invertible linear transformation between a finite sequence and coefficients on equally spaced discrete frequencies.
Discrete wavelet transform	Digital Signal ProcessingMultiresolution Wavelet Analysis	Decompose a sampled signal into multiscale approximation and detail coefficients through paired analysis filters and decimation, with reconstruction governed by the chosen wavelet filter bank and boundary convention.
Discrete-time Fourier transform	Signal Processing	A periodic continuous-frequency function obtained by summing a discrete-time sequence against complex exponentials, representing the sequence by its spectral amplitudes.
Discrete-to-continuum reference density	Continuous Entropy Reference	Carry declared discrete counting weights into a continuous reference for relative information entropy while stating the additive normalization and coordinate rule.
Elias Bassalygo bound	Coding Theory	An asymptotic upper bound on error-correcting-code rate derived by restricting codewords to a dense Hamming sphere and applying a Plotkin-type distance bound.
Entropy estimation	Entropy EstimationStatistical Estimation	In various science/engineering applications, such as independent component analysis, image analysis, genetic analysis, speech recognition, manifold learning, and time delay estimation it is useful to estimate the differential entropy of a system or process, given some observations.
Error exponents in hypothesis testing	Large DeviationsHypothesis Testing	Error exponents are sometimes referred to as error rates, due to the connection between hypothesis testing and information theory.
Error-Correcting Code	—	An error-correcting code is a specified set of valid codewords together with an encoding map, channel or error model, distance or recoverability structure, and decoding rule that introduces controlled redundancy so transmitted or stored information can be detected or reconstructed despite an admissible class of errors or erasures.
Error-Correcting Codes with Feedback	Feedback Channel Coding	Channel codes whose encoder causally adapts later transmissions to receiver-side information returned over a feedback link, with reliability, rate, delay, stopping, and feedback-noise assumptions stated as part of the code.
Even code	Coding TheoryBinary Linear Codes	A binary linear code in which every codeword has even Hamming weight; it is doubly even when all weights are divisible by four and strictly even when even but not doubly even.
EXIT Chart	Coding TheoryIterative Decoding	An extrinsic information transfer chart, commonly called an EXIT chart, is a technique to aid the construction of good iteratively-decoded error-correcting codes (in particular low-density parity-check (LDPC) codes and Turbo codes).
Expander code	Graph Based Codes	An expander code turns sparse local coding constraints on an expanding graph into global error-correction guarantees.
Filter (Signal Processing)	Signal ProcessingFilter Theory	A signal-to-signal system applies a specified response to selectively pass, attenuate, emphasize, delay, or estimate components of an input while producing a conditioned output signal.
Gabor wavelet	Signal Processing	A complex sinusoid localized by a Gaussian envelope, providing near-minimal joint uncertainty in position or time and spatial or temporal frequency.
Gambling and information theory	Kelly CriterionGambling and Investment	Kelly betting or proportional betting is an application of information theory to investing and gambling.
Generator matrix	Coding TheoryLinear Codes	In coding theory, a generator matrix is a matrix whose rows form a basis for a linear code.
Harmonic Wavelet Transform	Signal ProcessingHarmonic Wavelet Analysis	Partition a signal's Fourier coefficients into exact octave bands and inverse-transform each band into orthogonal, time-indexed harmonic-wavelet coefficients, preserving energy and exact reconstruction while exposing transient frequency content.
Hilbert spectral analysis	Signal ProcessingNonstationary Time Frequency Analysis	A time–frequency analysis that forms analytic signals, often from empirical mode components, and derives instantaneous amplitude and frequency from Hilbert phase.
Incompressibility method	Algorithmic Information Theory	A proof method that selects a Kolmogorov-incompressible object and shows that failure of the desired property would yield an impossibly shorter description.
Infomax	Neural ComputationSignal Processing	An information-theoretic design principle that selects an admissible input–output mapping by maximizing their mutual information under a stated probability model.
Information dimension	—	The asymptotic growth rate of the Shannon entropy of increasingly fine quantizations of a random variable or distribution.
Information Entropy	Discrete Information TheoryProbability	The probability-weighted average of logarithmic surprise across the outcomes of a discrete random variable.
Information projection	Divergence Minimization	Select from a constrained family the probability distribution minimizing a directed Kullback–Leibler divergence from a reference distribution, with direction and support kept explicit.
Kelly criterion	Gambling and Investment Theory	In finance, the Kelly criterion (or Kelly strategy or Kelly bet) is a formula for risk allocation with the sizing a sequence of bets by maximizing the long-term expected value of the logarithm of wealth, which is equivalent to maximizing the long-term expected geometric growth rate.
Kolmogorov complexity	Algorithmic Information Theory	The length of the shortest program for a fixed universal description language that outputs a given finite object and halts.
Krichevsky–Trofimov estimator	Universal Coding and Probability Estimation	Estimate categorical symbol probabilities by adding one-half to every observed count, the Jeffreys-prior predictive rule that attains asymptotically minimax worst-case coding regret.
Lag windowing	Signal ProcessingLinear Prediction and Autocorrelation Spectral Estimation	A stabilization technique that windows autocorrelation lags before estimating linear-prediction coefficients, thereby smoothing the power spectrum.
Lanczos resampling	Signal Processing	Signal resampling by convolution with a finite-windowed sinc kernel.
Linear programming decoding	Coding Theory	Error-correcting-code decoding by relaxing maximum-likelihood integer constraints to a tractable linear program over a codeword polytope approximation.
Log Gabor filter	Signal ProcessingSpatial Frequency Filters	A band-pass signal filter whose transfer function is Gaussian on a logarithmic frequency axis, providing no DC component and flexible bandwidth for localized multi-scale analysis.
Log-Sum Inequality	Information Inequalities	For nonnegative sequences aᵢ and bᵢ with totals A and B, the inequality ∑aᵢ log(aᵢ/bᵢ) ≥ A log(A/B), under standard zero conventions, with equality when the positive ratios aᵢ/bᵢ are constant.
Low-Density Parity-Check Code	Coding Theory	A linear error-correcting block code specified by sparse parity-check constraints on its valid codewords.
Lulu smoothing	Signal Processing	A nonlinear sequence smoother built from alternating local lower and upper envelope operators that remove impulsive noise while preserving step structure.
Mean square quantization error	Signal ProcessingQuantizer Distortion Analysis	Evaluate a quantizer by averaging the squared difference between each input and its reconstruction value under a declared input distribution, making large reconstruction deviations contribute quadratically.
Min-entropy	—	The negative logarithm of the largest outcome probability, measuring worst-case single-guess unpredictability as the order-infinity Rényi entropy.
Modified Discrete Cosine Transform	Signal ProcessingLapped Transform Coding	Map each 50%-overlapped real-signal block to a critically sampled cosine-coefficient block so that the apparent time-domain aliasing of each inverse block cancels when adjacent, compatibly windowed blocks are overlapped and added.
Modified Wigner distribution function	Signal ProcessingTime Frequency Distributions	A member of a modified quadratic time–frequency representation family designed to suppress the interference cross-terms of the Wigner distribution while retaining useful concentration.
Motion interpolation	Video Signal ProcessingFrame Rate Conversion	Synthesize temporally intermediate video frames by estimating motion between observed frames, mapping content toward the target time, and resolving occlusion and newly visible regions to increase temporal sampling.
Noisy-channel coding theorem	Channel CodingCoding Theory	Below a noisy channel's capacity, suitable long codes can make decoding error arbitrarily small.
Non-separable wavelet	Signal Processing	A multidimensional wavelet whose analyzing function or filter bank cannot be factored into tensor products of lower-dimensional wavelets.
Nonrecursive (FIR) Filter	Digital Signal ProcessingFinite Impulse Response Filtering	A causal discrete-time linear filter realized as a finite weighted sum of current and delayed input samples, with no previous-output feedback, so its finite tap vector completely determines both impulse and frequency response.
Oversampling	Signal Processing	Sampling a signal substantially above its Nyquist rate to ease antialias filtering, distribute quantization noise and improve effective resolution after filtering or decimation.
Parity-Check Matrix	Coding TheoryLinear Block Codes	A finite-field matrix whose linear check equations have exactly a specified linear code as their kernel and whose action on a received word yields an error syndrome.
Parvaresh–Vardy code	Coding Theory	An algebraic error-correcting code that encodes correlated polynomial evaluations so received words can be efficiently list-decoded beyond the Reed–Solomon radius.
Permutation Code	Coding TheoryPermutation Arrays	A codebook formed from permutations of n distinct symbols, equipped with a distance and minimum-separation requirement for detecting or correcting channel errors.
Phase response	Signal ProcessingFrequency Response	The frequency-dependent phase difference between a system's sinusoidal output and input, forming the angular component of its complex transfer function.
Polar code (coding theory)	Coding Theory	A linear error-correcting code that recursively transforms channels into nearly perfect and nearly useless bit-channels, placing information only on the reliable ones.
Polyphase matrix	Signal ProcessingMultirate Filter Banks	A matrix of polyphase filter components that represents a multirate filter bank, making downsampling structure, perfect reconstruction and efficient implementation algebraically explicit.
Quantum Game Theory	Quantum Information TheoryQuantum Strategies and Nonlocal Games	A strategic-interaction framework in which admissible strategies are quantum operations or measurements on specified quantum systems, with measurement-induced outcome probabilities and payoffs compared against a protocol-matched classical strategy set.
Quantum Mutual Information	Quantum Information Theory	In quantum information theory, quantum mutual information (QMI), or von Neumann mutual information, after John von Neumann, is a measure of correlation between subsystems of quantum state.
Quantum Operation	Quantum Information TheoryQuantum State Transformations	A completely positive, trace-nonincreasing linear transformation of quantum states whose output trace is the process or outcome probability and whose normalized output is the corresponding conditional state.
Quantum Relative Entropy	Quantum Information Theory	The asymmetric quantum divergence Tr[ρ(logρ−logσ)], finite under support inclusion and monotone under quantum channels, comparing an ordered pair of density operators.
Raptor code	Coding TheoryFountain Codes	In computer science, Raptor codes (rapid tornado; see Tornado codes) are the first known class of fountain codes with linear time encoding and decoding.
Rectangular function	Signal ProcessingElementary Signal Functions	A piecewise-constant pulse equal to one on a centered finite interval and zero outside, with a convention-dependent value at the endpoints.
Relay channel	Multi Terminal Communication	An information-theoretic channel with a source, a causally acting relay, and a destination, whose joint channel law determines what cooperation can achieve.
Repetition Code	Coding TheoryError Control	A block code that maps each symbol to a constant run of copies, trading rate for distance-based error recovery.
Sampling (signal processing)	Signal Processing	The representation of a continuous-domain signal by values taken at discrete time, space or other-domain locations.
Sequential decoding	Coding TheoryConvolutional Code Decoding	A variable-effort tree-search method for approximately maximum-likelihood decoding long convolutional or tree codes using far less memory than exhaustive trellis decoding.
Set redundancy compression	Data Compression	Compression that exploits shared structure across a collection of related data objects rather than encoding each object independently.
Shannon capacity of a graph	—	A graph invariant giving the asymptotic zero-error information rate of a confusability graph under repeated independent channel uses.
Shannon–Fano–Elias coding	—	A prefix-coding construction that chooses codewords from binary expansions of cumulative-probability midpoints.
Signal averaging	Signal Processing	The recovery of a repeatable signal by aligning repeated observations and averaging them so uncorrelated zero-mean noise cancels.
Signal Quantization	Signal ProcessingSource Coding	Assigning signal values to defined decision cells and recoverable representatives, with distortion assessed separately from the mapping.
Sophistication (complexity theory)	Algorithmic Information Theory	An algorithmic-information measure of the shortest model in which a string is a typical member at a declared significance tolerance.
Sparse Graph Code	Coding Theory	An error-correcting code whose sparse graph defines local parity-check or redundant-symbol generation relations for recovery under a stated error or erasure model.
Time–frequency analysis	Signal ProcessingNonstationary Signal Analysis	Represent a nonstationary signal jointly over time and frequency so changing spectral content, transients, and localization tradeoffs remain visible instead of being collapsed into one global spectrum.
Total variation denoising	Signal Processing	A signal-reconstruction method that balances fidelity to observed data against a penalty on total variation so noise is reduced while sharp edges are retained.
Trellis (graph)	Coding TheoryTrellis Coding	A trellis is a graph whose nodes are ordered into vertical slices (time) with every node at almost every time connected to at least one node at an earlier and at least one node at a later time.
Typical set	—	The high-probability set of long source sequences whose per-symbol information is close to the source entropy.
Variable bitrate	Data CompressionRate Control	An encoding mode that varies the number of bits assigned per time segment so complex content receives more data and simple content less at a targeted quality or average rate.
Wigner distribution function	Signal ProcessingBilinear Time Frequency Representation	Represent a signal or quantum state bilinearly over a conjugate-variable plane, preserving informative marginals and covariance while accepting negative values or interference cross-terms.
Wozencraft ensemble	Coding Theory	An explicit finite ensemble of rate-one-half linear codes over a finite field in which almost every member asymptotically meets the Gilbert–Varshamov distance bound.
Z-transform	Signal ProcessingDiscrete Time Transform Methods	Represent a discrete-time sequence by a Laurent series in a complex variable together with its region of convergence, enabling shifts, convolution, recurrences, spectra, and system behavior to be analyzed algebraically.
Zyablov Bound	Concatenated Codes	A proved achievability guarantee giving the binary rate–distance curve attainable by optimized single-level concatenated codes.
Zémor's Decoding Algorithm	Coding TheoryExpander Code Decoding	An alternating hard-decision decoder for Zémor expander codes that repeatedly projects all local edge words on one side of a regular bipartite graph to a nearest component-code word, then switches sides, with spectral expansion giving a logarithmic-round error-correction guarantee.