Probabilistic Machine Learning

Lecture schedule and reading

1. Kullback–Leibler is all you need

   • [slides1a.pdf] [slides1b.pdf]
   • Technical notes about KL divergence and overfitting [handout1.pdf]
   • Science and statistics, Box (1976) [pdf]
   • Statistical modeling: the two cultures, Breiman (2001) [pdf]

2. GANs and other adversaries

   • Pre-reading response due 19 Oct 4pm
   • Generative adversarial nets, Goodfellow et al. (2014)
   • f-GAN: training generative neural samplers using variational divergence minimization, Nowozin, Cseke and Tomioka (2016)
   • Revisiting classifier two-sample tests, Lopez-Paz and Oquab (2017)
   • Do deep generative models know what they don't know?, Nalisnick et al. (2019)

3. LLMs: the ghost in the machine

   • Direct preference optimization: your language model is secretly a reward model, Rafailov et al. (2023)
   • The pain axis: LLMs represent self-directed harm and act to relieve it, Tagliabue, Dung and Berg (2026)
   • Thought anchors: which LLM reasoning steps matter?, Bogdan et al. (2025)
   • Prompt baking, Bhargava et al. (2024)

4. VAEs: certificates of fit

   • Pre-reading response due 2 Nov 4pm
   • Visualizing data using t-SNE, van der Maaten and Hinton (2008)
   • Auto-encoding variational Bayes, Kingma and Welling (2014)
   • Probabilistic principal component analysis, Tipping and Bishop (1999) [pdf]

5. Diffusion and flows

   • Large language diffusion models, Nie et al. (2025)
   • Flow matching for generative modeling, Lipman et al. (2023)
   • Denoising diffusion probabilistic models, Ho, Jain and Abbeel (2020)

6. Random functions

   • Pre-reading response due 16 Nov 4pm
   • Conditional neural processes, Garnelo et al. (2018)
   • Autoregressive conditional neural processes, Bruinsma et al. (2023)
   • A temporal stochastic bias correction using a machine learning attention model, Nivron et al. (2024)

7. Takes on Uncertainty

   • Dropout as a Bayesian approximation: representing model uncertainty in deep learning, Gal and Ghahramani (2016)
   • Risk versus uncertainty in deep learning: Bayes, bootstrap and the dangers of dropout, Osband (2016) [pdf]
   • The Neural Testbed: evaluating joint predictions, Osband et al. (2022)
   • Why language models hallucinate, Kalai et al. (2025)

8. Causality as model fitting

   • Pre-reading response due 30 Nov 4pm
   • Invariant risk minimization, Arjovsky et al. (2019)
   • Causal inference by using invariant prediction: identification and confidence intervals, Peters, Bühlmann and Meinshausen (2016)
   • Potential outcome and directed acyclic graph approaches to causality: relevance for empirical practice in economics, Imbens (2020)