Week 10 — Bayesian Learning

Sahami 1998 Bayesian spam filter: every email service in the world running a variant. The Bayesian posture changes everything for uncertainty quantification.

Lecture

Naive Bayes as a baseline · Bayesian linear regression · conjugate priors · Gaussian processes at survey level · variational inference and MCMC at survey level.

Read before the lecture

• Murphy, *Probabilistic Machine Learning*, chapter 9 (Bayesian linear models)

Problem set

PS5 — Bayesian inference

1. Derive the posterior of Bayesian linear regression under a Gaussian prior.
2. Show that the posterior mean coincides with ridge regression under the right prior.
3. Implement Gibbs sampling for Bayesian linear regression on a small dataset.

Reference text for this week: chapter 10 of the bilingual notes — EN PDF · FR PDF.