Week 05 — Support Vector Machines

Vapnik 1992: the largest-margin classifier. The dominant method from 1995 to 2012.

Lecture

Hard-margin SVM as a quadratic program · soft-margin SVM (hinge loss) · the Lagrangian dual and the support-vector identification · the kernel trick · RBF, polynomial, sigmoid kernels.

Read before the lecture

• Boser, Guyon, Vapnik, *A Training Algorithm for Optimal Margin Classifiers* (COLT 1992)

Code lab

Lab 2 — SVM with a kernel sweep

Train SVMs with linear, polynomial, and RBF kernels on a public dataset. Tune $C$ and $\gamma$. Compare against logistic regression baselines. Discuss the kernel trick computational trade-off.

Notebook: lab02-svm.ipynb  ·  Dataset: Adult Census Income (UCI).

Reference text for this week: chapter 05 of the bilingual notes — EN PDF · FR PDF.