Fine-tuning Tools Comparison

Choose the right fine-tuning framework for training LLMs on Clore.ai GPU servers.

Fine-tuning adapts a pre-trained LLM to your specific task or domain. This guide compares the four leading open-source tools: Unsloth, Axolotl, LLaMA-Factory, and TRL — covering speed, memory efficiency, supported models, and ease of use.

Quick Decision Matrix

Unsloth

Axolotl

LLaMA-Factory

TRL

Best for

Speed + memory

Config-driven training

Beginner-friendly

Research + RLHF

Speed vs baseline

2-5× faster

~1× (standard)

Memory reduction

70-80% less

QLoRA standard

Standard

RLHF/DPO/PPO

Basic

✅

✅ (native)

WebUI

❌

✅

❌

GitHub stars

23K+

9K+

37K+

10K+

License

LGPL (free for non-commercial)

Apache 2.0

Overview

Unsloth

Unsloth is laser-focused on one thing: making fine-tuning as fast and memory-efficient as possible. It rewrites key operations in Triton and optimizes CUDA kernels.

Philosophy: Maximum speed, minimum VRAM — no compromises.

from unsloth import FastLanguageModel
import torch

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/Llama-3.2-8B-Instruct",
    max_seq_length=2048,
    load_in_4bit=True,  # 4-bit quantization
)

model = FastLanguageModel.get_peft_model(
    model,
    r=16,              # LoRA rank
    target_modules=["q_proj", "k_proj", "v_proj", "up_proj", "down_proj"],
    lora_alpha=16,
    lora_dropout=0,
    bias="none",
    use_gradient_checkpointing="unsloth",  # ~30% more batch size
    random_state=42,
)

Axolotl

Axolotl wraps HuggingFace Transformers with a YAML-based configuration system. It handles the complexity of training setup so you can focus on data and hyperparameters.

Philosophy: Everything in YAML, full flexibility underneath.

# config.yml
base_model: meta-llama/Meta-Llama-3-8B
model_type: LlamaForCausalLM
tokenizer_type: AutoTokenizer

datasets:
  - path: mhenrichsen/alpaca_data_cleaned
    type: alpaca

load_in_4bit: true
adapter: qlora

lora_r: 32
lora_alpha: 16
lora_target_modules:
  - q_proj
  - k_proj
  - v_proj
  - o_proj

num_epochs: 3
micro_batch_size: 2
gradient_accumulation_steps: 4
learning_rate: 2e-4

LLaMA-Factory

LLaMA-Factory supports the widest range of models (100+) and training methods, with a web UI for configuration. It's the most accessible option for non-researchers.

Philosophy: Everything works, for everyone.

# Train via command line
llamafactory-cli train \
  --model_name_or_path meta-llama/Meta-Llama-3-8B \
  --stage sft \
  --do_train \
  --dataset alpaca_gpt4_en \
  --template llama3 \
  --finetuning_type lora \
  --lora_rank 8 \
  --output_dir saves/llama3-8b-lora \
  --num_train_epochs 3.0 \
  --per_device_train_batch_size 2

# Or use WebUI
llamafactory-cli webui

TRL (Transformer Reinforcement Learning)

TRL is HuggingFace's official RLHF library. It's the standard for PPO, DPO, ORPO, and other alignment training methods.

Philosophy: Research-first, alignment training native.

from trl import SFTTrainer, SFTConfig
from transformers import AutoModelForCausalLM, AutoTokenizer
from datasets import load_dataset

model = AutoModelForCausalLM.from_pretrained("meta-llama/Meta-Llama-3-8B")
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3-8B")

training_args = SFTConfig(
    output_dir="./results",
    num_train_epochs=3,
    per_device_train_batch_size=2,
    gradient_accumulation_steps=4,
    learning_rate=2e-4,
    logging_steps=10,
)

trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    args=training_args,
    train_dataset=load_dataset("tatsu-lab/alpaca", split="train"),
)

trainer.train()

Speed Benchmarks

Training Speed Comparison (tokens/second)

Test setup: LLaMA 3.1 8B, LoRA r=16, 4-bit quantization, batch size 4, A100 80GB

Tool

Tokens/sec

vs Baseline

Memory (VRAM)

Unsloth (4-bit)

~4,200

2.8×

~8GB

Axolotl (QLoRA)

~1,500

1.0×

~16GB

LLaMA-Factory (QLoRA)

~1,480

~1.0×

~16GB

TRL (QLoRA)

~1,450

~0.97×

~18GB

Unsloth (full 16-bit)

~2,800

1.9×

~22GB

Unsloth advantage is real: 2-5× speed comes from custom Triton kernels for attention, cross-entropy, RoPE, and LoRA. Not just marketing.

VRAM Usage Comparison

Training LLaMA 3.1 8B, sequence length 2048:

Method

Unsloth

Axolotl

LLaMA-Factory

TRL

Full fine-tune (bf16)

60GB

70GB

72GB

74GB

LoRA (bf16)

18GB

24GB

25GB

26GB

QLoRA (4-bit)

8GB

16GB

18GB

QLoRA (4-bit, long ctx)

12GB

24GB

26GB

Minimum GPU for 8B model:

Unsloth: RTX 3080 (10GB) ✅
Others: RTX 3090 (24GB) required

Supported Models

Model Support Matrix

Model Family

Unsloth

Axolotl

LLaMA-Factory

TRL

LLaMA 3.x

✅

LLaMA 2

✅

Mistral

✅

Mixtral MoE

✅

Gemma 2

✅

Phi-3/3.5

✅

Qwen 2.5

✅

DeepSeek

✅

Falcon

✅

GPT-NeoX

Partial

✅

T5/FLAN

❌

✅

BERT/RoBERTa

❌

✅

Vision LLMs

Partial

✅

Training Method Support

Method

Unsloth

Axolotl

LLaMA-Factory

TRL

Full fine-tune

✅

LoRA

✅

QLoRA

✅

DoRA

✅

❌

PEFT

✅

SFT

✅

✅ (native)

DPO

✅

✅ (native)

PPO

❌

✅

✅ (native)

ORPO

✅

KTO

❌

✅

✅ (native)

GRPO

✅

❌

✅

CPT (continued pretraining)

✅

Unsloth: Deep Dive

What Makes It Fast

Triton kernels: Rewrites Flash Attention, cross-entropy loss, and LoRA in Triton
Fused operations: Combines multiple CUDA ops into one kernel
Smart gradient checkpointing: "unsloth" mode saves ~30% more memory
Efficient backprop: Avoids materializing large intermediate tensors

Installation on Clore.ai

# CUDA 12.1
pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
pip install --no-deps trl peft accelerate bitsandbytes

# Or with conda
conda create --name unsloth_env python=3.11
conda activate unsloth_env
conda install pytorch-cuda=12.1 pytorch cudatoolkit xformers -c pytorch -c nvidia -c xformers -y
pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
pip install --no-deps trl peft accelerate bitsandbytes

Complete Training Script

from unsloth import FastLanguageModel
from trl import SFTTrainer
from transformers import TrainingArguments
from datasets import load_dataset
import torch

# 1. Load model with Unsloth optimization
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/Meta-Llama-3.1-8B-Instruct",
    max_seq_length=2048,
    dtype=None,        # Auto-detect
    load_in_4bit=True,
)

# 2. Add LoRA adapter
model = FastLanguageModel.get_peft_model(
    model,
    r=16,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
                    "gate_proj", "up_proj", "down_proj"],
    lora_alpha=16,
    lora_dropout=0,
    bias="none",
    use_gradient_checkpointing="unsloth",
    random_state=3407,
)

# 3. Load and format dataset
dataset = load_dataset("tatsu-lab/alpaca", split="train")

def format_prompt(example):
    return {"text": f"### Instruction:\n{example['instruction']}\n\n### Response:\n{example['output']}"}

dataset = dataset.map(format_prompt)

# 4. Train
trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    train_dataset=dataset,
    dataset_text_field="text",
    max_seq_length=2048,
    args=TrainingArguments(
        per_device_train_batch_size=2,
        gradient_accumulation_steps=4,
        warmup_steps=5,
        num_train_epochs=1,
        learning_rate=2e-4,
        fp16=not torch.cuda.is_bf16_supported(),
        bf16=torch.cuda.is_bf16_supported(),
        logging_steps=1,
        optim="adamw_8bit",
        weight_decay=0.01,
        lr_scheduler_type="linear",
        seed=3407,
        output_dir="outputs",
    ),
)
trainer.train()

# 5. Save
model.save_pretrained("lora_model")
model.save_pretrained_gguf("model_gguf", tokenizer, quantization_method="q4_k_m")

Weaknesses: No PPO, limited to supported model list, LGPL license (check for commercial use)

Axolotl: Deep Dive

Configuration-First Approach

Axolotl shines when you want reproducible, version-controlled training configurations:

# axolotl_config.yml — full example
base_model: meta-llama/Meta-Llama-3-8B-Instruct
model_type: LlamaForCausalLM
tokenizer_type: AutoTokenizer

# Data
datasets:
  - path: tatsu-lab/alpaca
    type: alpaca
  - path: ./my_custom_data.jsonl
    type: sharegpt
dataset_prepared_path: ./prepared_data
val_set_size: 0.01

# Quantization
load_in_4bit: true
adapter: qlora
bf16: true
tf32: true

# LoRA
lora_r: 32
lora_alpha: 16
lora_dropout: 0.05
lora_target_modules:
  - q_proj
  - v_proj
  - k_proj
  - o_proj
  - gate_proj
  - up_proj
  - down_proj

# Training
sequence_len: 4096
sample_packing: true  # Packs short sequences for efficiency
pad_to_sequence_len: true
micro_batch_size: 2
gradient_accumulation_steps: 4
num_epochs: 3
learning_rate: 0.0002
optimizer: adamw_bnb_8bit
lr_scheduler: cosine

# Logging
logging_steps: 10
eval_steps: 100
save_steps: 100
output_dir: ./outputs/my-model

# wandb
wandb_project: my-fine-tune
wandb_run_id: run-001

# Install and run
pip install axolotl[flash-attn,deepspeed]
axolotl train axolotl_config.yml

Best for: Teams that want reproducible, config-versioned training runs

LLaMA-Factory: Deep Dive

WebUI Walkthrough

# Install
pip install llamafactory

# Launch WebUI
llamafactory-cli webui
# Open http://localhost:7860

WebUI tabs:

Train — configure base model, dataset, method
Evaluate — run MMLU, CMMLU benchmarks
Chat — interactive inference
Export — merge LoRA, quantize to GGUF

CLI Training Example

# Supervised Fine-Tuning
llamafactory-cli train \
  --stage sft \
  --model_name_or_path meta-llama/Meta-Llama-3-8B \
  --dataset alpaca_gpt4_en,glaive_toolcall_en \
  --template llama3 \
  --finetuning_type lora \
  --lora_rank 8 \
  --lora_alpha 16 \
  --lora_target all \
  --output_dir saves/llama3-lora \
  --num_train_epochs 3 \
  --per_device_train_batch_size 2 \
  --gradient_accumulation_steps 4 \
  --learning_rate 2e-4 \
  --quantization_bit 4 \
  --flash_attn fa2

# DPO Training
llamafactory-cli train \
  --stage dpo \
  --model_name_or_path meta-llama/Meta-Llama-3-8B \
  --dataset dpo_mix_en \
  --template llama3 \
  --finetuning_type lora \
  --output_dir saves/llama3-dpo

Best for: Beginners, teams wanting WebUI, DPO/RLHF without deep research knowledge

TRL: Deep Dive

RLHF Pipeline Example

TRL is the go-to for alignment training:

from trl import PPOTrainer, PPOConfig, AutoModelForCausalLMWithValueHead
from trl import DPOTrainer, DPOConfig
from datasets import load_dataset

# DPO (Direct Preference Optimization) — most common alignment method
model_name = "meta-llama/Meta-Llama-3-8B-Instruct"

dpo_config = DPOConfig(
    model_name_or_path=model_name,
    output_dir="dpo_outputs",
    num_train_epochs=1,
    per_device_train_batch_size=2,
    beta=0.1,             # KL penalty coefficient
    loss_type="sigmoid",  # or "hinge", "ipo", "kto_pair"
    learning_rate=5e-7,
)

# Load preference dataset (prompt + chosen + rejected)
dataset = load_dataset("Anthropic/hh-rlhf", split="train")

trainer = DPOTrainer(
    model=model_name,
    args=dpo_config,
    train_dataset=dataset,
)
trainer.train()

Best for: Alignment research, RLHF, DPO, PPO, ORPO implementations

Choosing the Right Tool

Decision Flow

Need maximum speed/minimum VRAM?
  → YES → Unsloth (2-5× faster, fits on smaller GPUs)

Need alignment training (DPO/PPO/RLHF)?
  → YES → TRL or LLaMA-Factory
  → Research/custom → TRL
  → Production/easy → LLaMA-Factory

Need configuration-first reproducibility?
  → YES → Axolotl

Non-technical team or want WebUI?
  → YES → LLaMA-Factory

Just want to start quickly?
  → LLaMA-Factory or Unsloth

By Team Type

Team

Recommendation

Reason

Individual researcher

Unsloth

Speed + Jupyter notebooks

ML engineer

Axolotl

Config-driven, reproducible

Product team

LLaMA-Factory

WebUI, wide model support

Alignment team

TRL

Native RLHF primitives

Startup

Unsloth + TRL

Speed + alignment when needed

Clore.ai GPU Recommendations

Task

Min GPU

Recommended

Tool

7-8B LoRA (QLoRA)

RTX 3080 (10GB)

RTX 3090

Unsloth

13B LoRA

RTX 3090 (24GB)

A6000 (48GB)

Unsloth/Axolotl

70B LoRA

A100 (80GB)

2×A100

Axolotl/TRL

8B Full FT

A100 (40GB)

A100 (80GB)

Any

DPO/PPO 7B

RTX 4090 (24GB)

A6000 (48GB)

TRL

Useful Links

Unsloth GitHub — 23K+ stars
Axolotl GitHub — 9K+ stars
LLaMA-Factory GitHub — 37K+ stars
TRL GitHub — 10K+ stars
HuggingFace PEFT Docs

Summary

Tool

Best for

Key advantage

Unsloth

Speed-critical training, small GPUs

2-5× faster, 70% less VRAM

Axolotl

Config-driven, reproducible runs

YAML-first, many data formats

LLaMA-Factory

100+ models, WebUI, beginners

Most model support, GUI

TRL

RLHF, DPO, alignment research

Native alignment training

For most Clore.ai use cases: start with Unsloth (speed + memory efficiency), add TRL if you need DPO or PPO alignment training.

PreviousVector Database Comparison NextVideo Generation Comparison

Last updated 23 hours ago

Was this helpful?

hashtagQuick Decision Matrix

hashtagOverview

hashtagUnsloth

hashtagAxolotl

hashtagLLaMA-Factory

hashtagTRL (Transformer Reinforcement Learning)

hashtagSpeed Benchmarks

hashtagTraining Speed Comparison (tokens/second)

hashtagVRAM Usage Comparison

hashtagSupported Models

hashtagModel Support Matrix

hashtagTraining Method Support

hashtagUnsloth: Deep Dive

hashtagWhat Makes It Fast

hashtagInstallation on Clore.ai

hashtagComplete Training Script

hashtagAxolotl: Deep Dive

hashtagConfiguration-First Approach

hashtagLLaMA-Factory: Deep Dive

hashtagWebUI Walkthrough

hashtagCLI Training Example

hashtagTRL: Deep Dive

hashtagRLHF Pipeline Example

hashtagChoosing the Right Tool

hashtagDecision Flow

hashtagBy Team Type

hashtagClore.ai GPU Recommendations

hashtagUseful Links

hashtagSummary

Quick Decision Matrix

Overview

Unsloth

Axolotl

LLaMA-Factory

TRL (Transformer Reinforcement Learning)

Speed Benchmarks

Training Speed Comparison (tokens/second)

VRAM Usage Comparison

Supported Models

Model Support Matrix

Training Method Support

Unsloth: Deep Dive

What Makes It Fast

Installation on Clore.ai

Complete Training Script

Axolotl: Deep Dive

Configuration-First Approach

LLaMA-Factory: Deep Dive

WebUI Walkthrough

CLI Training Example

TRL: Deep Dive

RLHF Pipeline Example

Choosing the Right Tool

Decision Flow

By Team Type

Clore.ai GPU Recommendations

Useful Links

Summary