MLflow

MLflow is an open-source platform for managing the complete Machine Learning lifecycle — from experiment tracking and model versioning to deployment and monitoring. Used by thousands of organizations worldwide, MLflow brings structure and reproducibility to ML workflows. Run it on Clore.ai's GPU cloud to get a centralized tracking server alongside your training jobs.

---

What is MLflow?

MLflow provides four core components:

Component	Description
Tracking	Log parameters, metrics, artifacts, and code from ML runs
Projects	Package code for reproducible runs
Models	Standard model format for deployment across frameworks
Model Registry	Centralized model store with versioning and lifecycle

Supported frameworks (built-in autologging):

• PyTorch, TensorFlow/Keras

• Scikit-learn, XGBoost, LightGBM

• HuggingFace Transformers

• Spark MLlib, statsmodels, Prophet

---

Prerequisites

Requirement	Value
GPU VRAM	Any (MLflow server itself is CPU-bound)
Storage	20 GB+ (for artifacts)
RAM	4 GB minimum for server
Ports	22 (SSH), 5000 (MLflow UI)

MLflow tracking server is lightweight. You can run it on a small CPU instance and point your GPU training jobs at it. Alternatively, co-locate it with your training GPU instance.

---

Step 1 — Rent a Server on Clore.ai

1. Log in to clore.ai.

2. Click Marketplace.

3. For a dedicated tracking server: filter by RAM ≥ 8 GB (GPU optional).

4. For co-located: use your existing training instance.

5. Set Docker image: ghcr.io/mlflow/mlflow:latest

6. Set open ports: 22 (SSH) and 5000 (MLflow UI).

7. Click Rent.

---

Step 2 — Launch the MLflow Tracking Server

The official ghcr.io/mlflow/mlflow image requires a startup command override.

In Clore.ai Docker Configuration

Set the command (or entrypoint override) to:

bash -c "apt-get update -q && apt-get install -y -q openssh-server && \
    mkdir /var/run/sshd && \
    echo 'root:clore123' | chpasswd && \
    sed -i 's/#PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config && \
    service ssh start && \
    mlflow server \
        --host 0.0.0.0 \
        --port 5000 \
        --default-artifact-root /mlflow/artifacts \
        --backend-store-uri sqlite:////mlflow/mlflow.db"

Alternative: Custom Dockerfile

FROM ghcr.io/mlflow/mlflow:latest

RUN apt-get update && apt-get install -y \
    openssh-server \
    && rm -rf /var/lib/apt/lists/*

# Configure SSH
RUN mkdir /var/run/sshd && \
    echo 'root:clore123' | chpasswd && \
    sed -i 's/#PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config

# Additional Python packages
RUN pip install boto3 psycopg2-binary

RUN mkdir -p /mlflow/artifacts

EXPOSE 22 5000

CMD service ssh start && \
    mlflow server \
        --host 0.0.0.0 \
        --port 5000 \
        --default-artifact-root /mlflow/artifacts \
        --backend-store-uri sqlite:////mlflow/mlflow.db

---

Step 3 — Access the MLflow UI

Open your browser:

http://<clore-host>:<public-port-5000>

You should see the MLflow Experiments dashboard.

The default SQLite backend (mlflow.db) stores all run metadata locally. For production or team use, switch to PostgreSQL — see Advanced Configuration below.

---

Step 4 — Log Your First Experiment

Connect from a Remote Training Job

On your training machine (or another Clore.ai instance), set the tracking URI:

export MLFLOW_TRACKING_URI=http://<clore-host>:<public-port-5000>

Basic PyTorch Experiment Logging

import mlflow
import mlflow.pytorch
import torch
import torch.nn as nn
import torch.optim as optim

# Connect to MLflow server
mlflow.set_tracking_uri("http://<clore-host>:<public-port-5000>")
mlflow.set_experiment("my-first-experiment")

# Define a simple model
class SimpleNet(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super().__init__()
        self.fc1 = nn.Linear(input_size, hidden_size)
        self.relu = nn.ReLU()
        self.fc2 = nn.Linear(hidden_size, output_size)
    
    def forward(self, x):
        return self.fc2(self.relu(self.fc1(x)))

# Training with MLflow tracking
with mlflow.start_run(run_name="training-run-001"):
    # Log hyperparameters
    params = {
        "learning_rate": 0.001,
        "batch_size": 64,
        "epochs": 100,
        "hidden_size": 256,
        "optimizer": "adam"
    }
    mlflow.log_params(params)
    
    # Initialize model
    model = SimpleNet(784, 256, 10).cuda()
    optimizer = optim.Adam(model.parameters(), lr=params["learning_rate"])
    criterion = nn.CrossEntropyLoss()
    
    # Training loop
    for epoch in range(params["epochs"]):
        loss = torch.tensor(0.5 / (epoch + 1))  # Simulated
        accuracy = 0.7 + epoch * 0.003
        
        # Log metrics at each epoch
        mlflow.log_metrics({
            "train_loss": loss.item(),
            "train_accuracy": accuracy,
        }, step=epoch)
    
    # Log the final model
    mlflow.pytorch.log_model(model, "model")
    
    # Log final metrics
    mlflow.log_metric("final_accuracy", accuracy)
    
    print(f"Run logged to MLflow. ID: {mlflow.active_run().info.run_id}")

HuggingFace Transformers Autologging

import mlflow
from transformers import TrainingArguments, Trainer

mlflow.set_tracking_uri("http://<clore-host>:<public-port-5000>")
mlflow.set_experiment("llm-finetuning")

# Enable autologging — automatically logs params, metrics, and model
mlflow.transformers.autolog()

training_args = TrainingArguments(
    output_dir="./results",
    num_train_epochs=3,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=64,
    learning_rate=2e-5,
    warmup_steps=500,
    weight_decay=0.01,
    logging_dir="./logs",
    logging_steps=10,
    evaluation_strategy="epoch",
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset,
)

with mlflow.start_run():
    trainer.train()

---

Step 5 — Scikit-learn with Autologging

import mlflow
import mlflow.sklearn
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.datasets import load_digits

mlflow.set_tracking_uri("http://<clore-host>:<public-port-5000>")
mlflow.set_experiment("sklearn-experiments")

# Autolog everything
mlflow.sklearn.autolog()

X, y = load_digits(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

with mlflow.start_run(run_name="random-forest-v1"):
    rf = RandomForestClassifier(n_estimators=100, max_depth=10, random_state=42)
    rf.fit(X_train, y_train)
    
    score = rf.score(X_test, y_test)
    print(f"Test Accuracy: {score:.4f}")
    # All params, metrics, and model automatically logged!

---

Step 6 — Model Registry

Register and manage model versions via the UI or API:

import mlflow

client = mlflow.MlflowClient("http://<clore-host>:<public-port-5000>")

# Register a model from a run
run_id = "your-run-id-here"
model_uri = f"runs:/{run_id}/model"

registered = mlflow.register_model(
    model_uri=model_uri,
    name="production-classifier"
)

print(f"Version: {registered.version}")

# Transition model stage
client.transition_model_version_stage(
    name="production-classifier",
    version=registered.version,
    stage="Production"
)

# Load a production model anywhere
model = mlflow.pyfunc.load_model(
    model_uri="models:/production-classifier/Production"
)

---

Step 7 — Serve a Model

MLflow can serve any logged model as a REST API:

# On the MLflow server instance
export MLFLOW_TRACKING_URI=http://localhost:5000

mlflow models serve \
    --model-uri "models:/production-classifier/Production" \
    --host 0.0.0.0 \
    --port 5001 \
    --no-conda

Test the served model:

curl -X POST http://<clore-host>:5001/invocations \
    -H "Content-Type: application/json" \
    -d '{"inputs": [[1.0, 2.0, 3.0, ...]]}'

---

Advanced Configuration

PostgreSQL Backend (Production)

# Launch with PostgreSQL
mlflow server \
    --host 0.0.0.0 \
    --port 5000 \
    --backend-store-uri postgresql://user:password@db-host/mlflow \
    --default-artifact-root s3://my-bucket/mlflow-artifacts

S3 Artifact Store

pip install boto3

export AWS_ACCESS_KEY_ID=your_key
export AWS_SECRET_ACCESS_KEY=your_secret

mlflow server \
    --host 0.0.0.0 \
    --port 5000 \
    --default-artifact-root s3://my-mlflow-bucket/artifacts \
    --backend-store-uri sqlite:////mlflow/mlflow.db

Authentication (Enterprise)

pip install mlflow[auth]

mlflow server \
    --host 0.0.0.0 \
    --port 5000 \
    --app-name basic-auth \
    --backend-store-uri sqlite:////mlflow/mlflow.db \
    --default-artifact-root /mlflow/artifacts

---

Comparing Runs in the UI

1. Open the MLflow UI at http://<clore-host>:<port>

2. Select an experiment from the left panel

3. Check the boxes next to multiple runs

4. Click Compare to see side-by-side metrics and parameters

5. Use the Charts tab for visual comparison

---

Troubleshooting

Cannot Connect to Tracking Server

mlflow.exceptions.MlflowException: API request failed with status code 503

Solutions:

• Check that port 5000 is open and forwarded in Clore.ai

• Verify the server is running: ps aux | grep mlflow

• Test connectivity: curl http://<clore-host>:<port>/health

Artifact Upload Fails

Solution: Ensure the artifact directory is writable:

chmod 777 /mlflow/artifacts

SQLite Locked Error (Concurrent Writes)

Solution: Switch to PostgreSQL for multi-user setups:

pip install psycopg2-binary

Model Registry Not Showing

Solution: Verify you're using a --backend-store-uri that supports the registry (SQLite or PostgreSQL — not just a local path).

---

Cost Estimation

Instance	Use Case	Est. Price	Notes
CPU 4-core	Tracking server only	~$0.05/hr	Very lightweight
RTX 3080	Co-located training	~$0.10/hr	Training + MLflow
RTX 4090	Heavy training + tracking	~$0.35/hr	Most common setup

Run MLflow on a cheap CPU instance and point all your GPU training jobs at it. This way the tracking server runs continuously without burning expensive GPU credits.

---

Useful Resources

• MLflow Official Documentation

• MLflow GitHub

• MLflow Docker Hub

• MLflow Model Registry Guide

• MLflow Tracking API Reference

---

Clore.ai GPU Recommendations

Use Case	Recommended GPU	Est. Cost on Clore.ai
Development/Testing	RTX 3090 (24GB)	~$0.12/gpu/hr
Production Training	RTX 4090 (24GB)	~$0.70/gpu/hr
Large Scale Experiments	A100 80GB	~$1.20/gpu/hr

💡 All examples in this guide can be deployed on Clore.ai GPU servers. Browse available GPUs and rent by the hour — no commitments, full root access.