Generative & Efficient Neural Architectures

← Back to Domain-Specific Families

Abstractions about training and compressing neural network models, covering generative-modeling frameworks (Generative Adversarial Network, Deep Belief Network, Discrete Diffusion Model), their failure modes like Mode Collapse, and efficiency-oriented architectures and techniques such as MobileNet, Model Compression and Data Augmentation.

7 abstractions in this family — domain-specific abstractions that sit near one another in structural-signature space (k-means over structural-signature embeddings). Each is shown with its short description.

• Data Augmentation — Data augmentation is a statistical technique which allows maximum likelihood estimation from incomplete data.
• Deep belief network — A multilayer generative model whose top layers form an undirected associative model and lower layers form directed latent-variable connections.
• Discrete diffusion model — A latent-variable generative model that corrupts categorical states through a forward Markov jump process and learns a reverse process to generate data.
• Generative adversarial network — A generative-learning framework in which a generator and discriminator are trained in opposition so generated samples approach a data distribution.
• MobileNet — A family of convolutional neural-network architectures designed for efficient vision inference through depthwise-separable convolutions and later mobile-oriented bottleneck, activation, attention, and scaling designs.
• Mode collapse — A generative-model failure in which the learned output distribution represents too few modes and produces markedly reduced diversity.
• Model compression — The reduction of a trained machine-learning model’s storage, memory or computation while preserving an explicitly tolerated level of task performance.