Home » Notes » Machine Learning

Generative Models

Notes on VAEs, diffusion models, GANs, and other generative approaches. Covers theory, architectures, and training techniques for generation tasks.
Machine Learning Fundamentals
Visualization of inverse problem showing one input mapping to multiple valid outputs

Mixture Density Networks

Seminal 1994 paper introducing MDNs to model arbitrary conditional probability distributions using neural networks.

Generative Modeling
Diagram comparing standard stochastic sampling (gradient blocked) vs the reparameterization trick (gradient flows)

Auto-Encoding Variational Bayes (VAE Paper Summary)

Summary of Kingma & Welling's foundational VAE paper introducing the reparameterization trick and variational …

Generative Modeling
Flowchart comparing VAE and IWAE computation showing the key difference in where averaging occurs relative to the log operation

IWAE: Importance Weighted Autoencoders

Summary of Burda, Grosse & Salakhutdinov's ICLR 2016 paper introducing Importance Weighted Autoencoders for tighter …

Generative Modeling
Visualization of the VAE prior hole problem showing a ring-shaped aggregate posterior with an empty center where the Gaussian prior has highest density

Contrastive Learning for Variational Autoencoder Priors

Aneja et al.'s NeurIPS 2021 paper introducing Noise Contrastive Priors (NCPs) to address VAE's 'prior hole' problem with …