Experiment Tracking with MLflow

Introduction

Experiment tracking is the process of recording and managing all information about machine learning experiments, such as model parameters, metrics, and results. MLflow is a powerful open-source platform that simplifies this process by providing easy tracking, reproducibility, and deployment.

بالعربية المغربية (الدارجة): تتبع التجارب فالتعلم الآلي هو عملية تسجيل كل المعلومات ديال التجارب بحال المعاملات، النتائج، والمقاييس. MLflow هو منصة مفتوحة كيسهّل هاد الخدمة وكيخلي التجارب منظمة وسهلة الإعادة.

Why Experiment Tracking Matters

• Ensures reproducibility of experiments.
• Helps compare different models easily.
• Keeps track of hyperparameters and results.
• Facilitates collaboration in data teams.
• Supports model versioning and deployment.

بالعربية المغربية: التتبع ديال التجارب كيساعد باش نرجعو لأي تجربة قديمة، نقارنو النماذج، ونعرفو شنو أحسن إعدادات بدون ما نضيعو الوقت.

Core Concepts of MLflow

• Run: A single experiment execution where metrics and parameters are logged.
• Experiment: A collection of related runs under one project.
• Artifact: Files like models, plots, or logs stored for each run.
• Tracking Server: A centralized service to manage experiments and results.

Setting Up MLflow

# Install MLflow
pip install mlflow

# Start MLflow tracking server locally
mlflow ui

This command opens the MLflow user interface at http://localhost:5000 where you can view all your experiments.

بالعربية المغربية: من بعد التثبيت، نقدر نفتحو واجهة MLflow على الرابط http://localhost:5000 باش نشوفو كل التجارب ديالنا بشكل منظم.

Python Example: Logging Experiments with MLflow

import mlflow
import mlflow.sklearn
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split

# Load data
X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Define experiment
mlflow.set_experiment("iris-rf-experiment")

# Start run
with mlflow.start_run():
    n_estimators = 100
    max_depth = 5
    model = RandomForestClassifier(n_estimators=n_estimators, max_depth=max_depth, random_state=42)
    model.fit(X_train, y_train)
    accuracy = model.score(X_test, y_test)
    
    # Log parameters and metrics
    mlflow.log_param("n_estimators", n_estimators)
    mlflow.log_param("max_depth", max_depth)
    mlflow.log_metric("accuracy", accuracy)
    
    # Save model
    mlflow.sklearn.log_model(model, "model")

print("Experiment logged successfully.")

Explanation of the Example

• The experiment is named iris-rf-experiment.
• Model parameters and accuracy are recorded using mlflow.log_param() and mlflow.log_metric().
• The model is saved automatically as an artifact.
• All results can be viewed on the MLflow UI.

بالعربية المغربية: هاد الكود كيدير تجربة جديدة بالإسم iris-rf-experiment، كيسجّل المعاملات بحال عدد الأشجار والعمق، وكيحسب الدقة، ومن بعد كيسجّل النموذج فالواجهة ديال MLflow.

Comparing Multiple Experiments

MLflow lets you compare different runs visually. You can see parameter values, metrics, and charts side by side to pick the best model.

بالعربية المغربية: فـMLflow نقدر نقارنو بين التجارب باش نعرفو شنو النموذج اللي خدم مزيان، وكل تجربة كتكون فيها النتائج والمقاييس بشكل واضح.

Advanced MLflow Features

• MLflow Projects: Package code and environment for reproducibility.
• MLflow Models: Manage model deployment formats.
• MLflow Registry: Store and version trained models.
• MLflow UI: Visualize all experiments in one dashboard.

Best Practices for Experiment Tracking

• Always log both parameters and metrics.
• Tag each experiment with dataset and model version.
• Use consistent experiment naming conventions.
• Regularly clean unused runs to save storage.
• Automate experiment logging in your training pipeline.

Example: Registering Models in MLflow

# After logging a model, you can register it
result = mlflow.register_model(
    "runs:/<run_id>/model",
    "IrisModelRegistry"
)

print("Model registered with version:", result.version)

بالعربية المغربية: من بعد التجربة نقدر نسجلو النموذج فالـModel Registry باش نستعملوه فالتطبيق أو التحديث القادم.

10 Exercises for Practice

1. Define what experiment tracking is and why it’s important.
2. Install MLflow and start the local tracking server.
3. Run the provided example and view results in the MLflow UI.
4. Add more hyperparameters (like min_samples_split) to your logged model.
5. Log confusion matrix as an artifact in MLflow.
6. Compare two models using different hyperparameters.
7. Export experiment results as a CSV file from the MLflow UI.
8. Register your best model using MLflow Model Registry.
9. Integrate MLflow tracking with your existing ML pipeline.
10. Explain the difference between MLflow Tracking, Projects, and Models.