MLflow for Experiment Tracking in Student Projects

Every ML student knows this moment: you trained twelve model variants, the winning one is somewhere in a notebook named model_final_FINAL2.ipynb, and you cannot remember which learning rate produced it. Experiment tracking is the discipline — and tooling — that ends this chaos: every run's parameters, metrics, code version, and artifacts recorded automatically, comparable side by side.

MLflow is the standard open-source tool for this. This guide covers what to track, how to set up MLflow for a student project, and the habits that make your experiments reproducible — including the ones your project report will need.

What experiment tracking actually records

A training run produces more than a model file. To reproduce or understand it later, you need:

What	Example	Why it matters
Parameters	lr=1e-3, batch=32, epochs=50	The knobs you turned
Metrics	val_accuracy=0.87 at epoch 50	What happened, over time
Code version	git commit hash	Which code produced this
Data version	dataset hash or version tag	Which data produced this
Artifacts	model weights, confusion matrix plot, sample predictions	The outputs worth keeping
Environment	library versions	"It worked on my machine" insurance

Note: The data version is the one students skip and regret most. A model trained on dataset-v2 compared against metrics from dataset-v1 is a meaningless comparison — and you will not notice until report-writing week.

Setting up MLflow in five minutes

MLflow Tracking runs as a local server with a web UI. Install, start, point your code at it:

pip install mlflow
mlflow ui --port 5000

Then in your training script:

import mlflow

mlflow.set_tracking_uri("http://localhost:5000")
mlflow.set_experiment("plant-disease-classifier")

with mlflow.start_run(run_name="mobilenet-lr1e3"):
    mlflow.log_param("learning_rate", 1e-3)
    mlflow.log_param("batch_size", 32)
    # ... training loop ...
    mlflow.log_metric("val_accuracy", 0.87)
    mlflow.log_artifact("confusion_matrix.png")

Open localhost:5000 and every run appears as a row: parameters, metrics, artifacts, all comparable and sortable. That UI replaces your spreadsheet of results — the one you stopped updating in week three.

The autolog shortcut

For scikit-learn, PyTorch (via pytorch-lightning), XGBoost, and others, MLflow can log automatically:

mlflow.sklearn.autolog()   # logs params, metrics, and the model itself

Autolog is excellent for classical ML and fine for early deep-learning experiments. For serious training loops, explicit logging (log_param/log_metric per epoch) gives cleaner metric histories and avoids surprises. Use autolog to start, graduate to explicit logging when you care about the details.

What to log: a student-project checklist

For every run, log these without exception:

• All hyperparameters (learning rate, batch size, optimizer, scheduler, epochs, seed)
• Dataset identifier and split sizes (train/val/test counts)
• Model architecture name and any config (layers, dropout, pretrained backbone)
• Per-epoch training and validation metrics (not just the final number)
• Random seed — without it, the run is not reproducible
• Git commit hash of the training code
• Final test-set metrics, computed once, on data never used for tuning
• The model artifact itself (or a path to it)
• A confusion matrix or error examples for classification tasks

The per-epoch history matters: two runs can end at the same accuracy with completely different learning dynamics, and the curves tell you which one to trust.

Comparing runs: how decisions get made

The MLflow UI's comparison view is where tracking pays off. Select runs, compare metric curves side by side, and answer questions like:

• Did the learning-rate change help, or did it just get lucky? (Check multiple seeds.)
• Is the model still improving at epoch 50, or did it plateau at 30? (Early stopping decision.)
• Does the bigger model beat the smaller one on validation, or only on training? (Overfitting check — see the overfitting guide.)

A practical habit: one variable per run group. Change the learning rate OR the architecture OR the augmentation — never all three. MLflow makes the comparison easy; discipline makes it meaningful.

MLflow Models and the deployment handoff

MLflow can package a trained model in a standard format (mlflow.sklearn.log_model, mlflow.pytorch.log_model) with its dependencies recorded. This turns "here's a .pth file, good luck" into a loadable artifact:

model = mlflow.pytorch.load_model("runs:/<run_id>/model")

For deployment, this pairs naturally with the model registry concept (staging → production tags). Student projects rarely need the full registry ceremony, but logging the model artifact with its run context means your deployment references a specific, reproducible experiment — not a mystery file.

Reproducibility: the unglamorous multiplier

Tracking is half of reproducibility; the other half is determinism:

• Fix seeds for Python, NumPy, and your framework — and log the seed value.
• Pin dependencies (pip freeze into the run's artifacts or a requirements file).
• Version your data. Even a simple convention — data/v1/, data/v2/ folders with a README of what changed — beats nothing.
• One command to reproduce. A training script runnable as python train.py --config config.yaml with the config logged to MLflow. Notebooks are fine for exploration; the final runs should be scripts.

When your examiner asks "how did you get this number," the answer should be a run ID, not a memory.

Common mistakes

• Tracking only the final metric. Without per-epoch curves you cannot diagnose anything.
• Logging test metrics during tuning. The test set is for one final evaluation. Tune on validation; report on test — once.
• No seeds. Every run is a snowflake; nothing is comparable.
• Comparing runs across different data versions. Log the dataset version or the comparison is fiction.
• Metric name chaos. val_acc, validation_accuracy, and acc_val across runs do not compare. Fix metric names on day one.
• Treating tracking as report-season work. Backfilling runs from memory produces fiction. Log from the first experiment.

Alternatives worth knowing

Tool	Character
MLflow	Self-hosted, full lifecycle (tracking + models + registry), free
Weights & Biases	Hosted, beautiful UI, generous free tier for individuals
TensorBoard	Great for metric curves, weaker for parameter/artifact bookkeeping
Plain CSV + git	Works for tiny projects; collapses under real experiment counts

Any of these beats notebooks-and-memory. MLflow's advantage for students: free, local, no account, and the skills transfer to industry where it is widely used.

Where to go from here

Tracking supports the whole modeling workflow: pair it with the overfitting fix guide when curves look wrong, the ML evaluation metrics guide for choosing what to log, and data augmentation as the experiments you will track. For packaging finished models, see Docker for student projects. More ML workflow topics in the AI & Machine Learning branch hub.