Lab 3 — Instrumented training loop

Goal. Take any training script from a prior course. Add MLflow tracking. Run a hyperparameter sweep. Promote the best model to the registry.

What you ship. MLflow run history (screenshot or exported HTML), the best model in the registry tagged Production, and a 200-word note on what surprised you in the sweep.

Setup

Install the dependencies (one-time).

# !pip install mlflow scikit-learn pandas

import mlflow
import mlflow.sklearn
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, roc_auc_score
import numpy as np
import os

os.environ.setdefault('MLFLOW_TRACKING_URI', 'sqlite:///mlflow.db')

A small but real dataset to sweep over

data = load_breast_cancer()
X_train, X_test, y_train, y_test = train_test_split(data.data, data.target, test_size=0.2, random_state=42)
print('train/test shapes:', X_train.shape, X_test.shape)

Exercise 1 — Instrument a single training run

with mlflow.start_run():
    mlflow.log_param('n_estimators', 100)
    clf = RandomForestClassifier(n_estimators=100, random_state=42).fit(X_train, y_train)
    p = clf.predict_proba(X_test)[:, 1]
    mlflow.log_metric('auc', roc_auc_score(y_test, p))
    mlflow.sklearn.log_model(clf, 'model')

Exercise 2 — Hyperparameter sweep

# YOUR TURN
# Sweep n_estimators in {50, 100, 200, 500} and max_depth in {3, 5, 10, None}.
# Log each run to MLflow.

Exercise 3 — Promote the best run

# YOUR TURN
# Identify the best run by AUC. Register it. Transition to Production.

Done?

Submit per the cohort schedule. Peer review pairing announced the following Monday.