Fine-tune LLM

Train your own custom LLM using efficient fine-tuning techniques on CLORE.AI GPUs.

All examples can be run on GPU servers rented through CLORE.AI Marketplace.

Renting on CLORE.AI

Visit CLORE.AI Marketplace
Filter by GPU type, VRAM, and price
Choose On-Demand (fixed rate) or Spot (bid price)
Configure your order:
- Select Docker image
- Set ports (TCP for SSH, HTTP for web UIs)
- Add environment variables if needed
- Enter startup command
Select payment: CLORE, BTC, or USDT/USDC
Create order and wait for deployment

Access Your Server

Find connection details in My Orders
Web interfaces: Use the HTTP port URL
SSH: ssh -p <port> root@<proxy-address>

What is LoRA/QLoRA?

LoRA (Low-Rank Adaptation) - Train small adapter layers instead of full model
QLoRA - LoRA with quantization for even less VRAM
Train 7B model on single RTX 3090
Train 70B model on single A100

Requirements

Model

Method

Min VRAM

Recommended

QLoRA

12GB

RTX 3090

13B

QLoRA

20GB

RTX 4090

70B

QLoRA

48GB

A100 80GB

Full LoRA

24GB

RTX 4090

Quick Deploy

Docker Image:

pytorch/pytorch:2.5.1-cuda12.4-cudnn9-devel

Ports:

22/tcp
8888/http
6006/http

Command:

pip install transformers datasets accelerate peft bitsandbytes trl wandb jupyterlab && \
jupyter lab --ip=0.0.0.0 --port=8888 --allow-root

Accessing Your Service

After deployment, find your http_pub URL in My Orders:

Go to My Orders page
Click on your order
Find the http_pub URL (e.g., abc123.clorecloud.net)

Use https://YOUR_HTTP_PUB_URL instead of localhost in examples below.

Dataset Preparation

Chat Format (Recommended)

[
  {
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "What is Python?"},
      {"role": "assistant", "content": "Python is a programming language..."}
    ]
  }
]

Instruction Format

[
  {
    "instruction": "Translate to French",
    "input": "Hello, how are you?",
    "output": "Bonjour, comment allez-vous?"
  }
]

Alpaca Format

[
  {
    "instruction": "Give three tips for staying healthy.",
    "input": "",
    "output": "1. Eat balanced meals..."
  }
]

QLoRA Fine-tuning Script

import torch
from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
    BitsAndBytesConfig,
    TrainingArguments,
)
from peft import LoraConfig, get_peft_model, prepare_model_for_kbit_training
from datasets import load_dataset
from trl import SFTTrainer

# === Configuration ===
MODEL_NAME = "mistralai/Mistral-7B-v0.1"
DATASET = "your_dataset.json"  # or HuggingFace dataset name
OUTPUT_DIR = "./output"
MAX_SEQ_LENGTH = 2048

# === Load Model with 4-bit Quantization ===
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)

model = AutoModelForCausalLM.from_pretrained(
    MODEL_NAME,
    quantization_config=bnb_config,
    device_map="auto",
    trust_remote_code=True,
)

tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "right"

# === Configure LoRA ===
lora_config = LoraConfig(
    r=64,                    # Rank (higher = more capacity, more VRAM)
    lora_alpha=16,           # Scaling factor
    target_modules=[
        "q_proj", "k_proj", "v_proj", "o_proj",  # Attention
        "gate_proj", "up_proj", "down_proj",      # MLP
    ],
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM",
)

# Prepare model for training
model = prepare_model_for_kbit_training(model)
model = get_peft_model(model, lora_config)

# Print trainable parameters
model.print_trainable_parameters()

# === Load Dataset ===
dataset = load_dataset("json", data_files=DATASET)

# Or: dataset = load_dataset("timdettmers/openassistant-guanaco")

# === Format Dataset ===
def format_chat(example):
    # Adjust based on your dataset format
    text = f"### Instruction:\n{example['instruction']}\n\n"
    if example.get('input'):
        text += f"### Input:\n{example['input']}\n\n"
    text += f"### Response:\n{example['output']}"
    return {"text": text}

dataset = dataset.map(format_chat)

# === Training Arguments ===
training_args = TrainingArguments(
    output_dir=OUTPUT_DIR,
    num_train_epochs=3,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=4,
    learning_rate=2e-4,
    weight_decay=0.001,
    warmup_ratio=0.03,
    lr_scheduler_type="cosine",
    logging_steps=10,
    save_steps=100,
    save_total_limit=3,
    fp16=True,
    optim="paged_adamw_8bit",
    max_grad_norm=0.3,
    group_by_length=True,
    report_to="wandb",  # or "tensorboard"
)

# === Train ===
trainer = SFTTrainer(
    model=model,
    train_dataset=dataset["train"],
    dataset_text_field="text",
    max_seq_length=MAX_SEQ_LENGTH,
    tokenizer=tokenizer,
    args=training_args,
    packing=True,  # Pack multiple examples into one sequence
)

trainer.train()

# === Save ===
trainer.save_model(f"{OUTPUT_DIR}/final")
tokenizer.save_pretrained(f"{OUTPUT_DIR}/final")

Using Axolotl (Easier)

Axolotl simplifies fine-tuning with YAML configs:

pip install axolotl

# Create config
cat > config.yml << 'EOF'
base_model: mistralai/Mistral-7B-v0.1
model_type: MistralForCausalLM
tokenizer_type: LlamaTokenizer

load_in_4bit: true
adapter: qlora
lora_r: 32
lora_alpha: 16

datasets:
  - path: your_data.json
    type: alpaca

sequence_len: 4096
sample_packing: true

gradient_accumulation_steps: 4
micro_batch_size: 2
num_epochs: 3
learning_rate: 2e-4

output_dir: ./output
EOF

# Train
accelerate launch -m axolotl.cli.train config.yml

Axolotl Config Examples

Chat Model

base_model: mistralai/Mistral-7B-Instruct-v0.2
load_in_4bit: true
adapter: qlora

datasets:
  - path: data.json
    type: sharegpt

chat_template: mistral

Code Model

base_model: codellama/CodeLlama-7b-hf
load_in_4bit: true
adapter: qlora

datasets:
  - path: code_data.json
    type: alpaca

sequence_len: 8192  # Longer context for code

Merging LoRA Weights

After training, merge LoRA back into base model:

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "mistralai/Mistral-7B-v0.1",
    torch_dtype=torch.float16,
    device_map="auto",
)

# Load LoRA
model = PeftModel.from_pretrained(base_model, "./output/final")

# Merge
merged_model = model.merge_and_unload()

# Save merged model
merged_model.save_pretrained("./merged_model")
tokenizer.save_pretrained("./merged_model")

Convert to GGUF

For use with llama.cpp/Ollama:


# Clone llama.cpp
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp

# Convert
python convert.py ../merged_model --outtype f16 --outfile model-f16.gguf

# Quantize
./quantize model-f16.gguf model-q4_k_m.gguf q4_k_m

Monitoring Training

Weights & Biases

import wandb
wandb.init(project="llm-finetune", name="mistral-7b-lora")

TensorBoard


# In training args
report_to="tensorboard"
logging_dir="./logs"

# View
tensorboard --logdir ./logs --port 6006 --bind_all

Best Practices

Hyperparameters

Parameter

7B Model

13B Model

70B Model

batch_size

grad_accum

2e-4

1e-4

5e-5

lora_r

epochs

2-3

1-2

Dataset Size

Minimum: 1,000 examples
Good: 10,000+ examples
Quality > Quantity

Avoiding Overfitting

training_args = TrainingArguments(
    ...
    weight_decay=0.01,
    warmup_ratio=0.03,
    save_total_limit=3,
    load_best_model_at_end=True,
    evaluation_strategy="steps",
    eval_steps=100,
)

Multi-GPU Training


# With accelerate
accelerate launch --multi_gpu --num_processes 4 train.py

# With DeepSpeed
accelerate launch --use_deepspeed --num_processes 4 train.py

DeepSpeed config:

{
  "bf16": {"enabled": true},
  "zero_optimization": {
    "stage": 2,
    "offload_optimizer": {"device": "cpu"}
  }
}

Saving & Exporting


# Save LoRA adapter
trainer.save_model("./lora_adapter")

# Save merged model
merged_model.save_pretrained("./full_model")

# Upload to HuggingFace
huggingface-cli login
merged_model.push_to_hub("username/my-model")

Troubleshooting

OOM Errors

Reduce batch size
Increase gradient accumulation
Use gradient_checkpointing=True
Reduce lora_r

Training Loss Not Decreasing

Check data format
Increase learning rate
Check for data issues

NaN Loss

Reduce learning rate
Use fp32 instead of fp16
Check for corrupted data

Cost Estimate

Typical CLORE.AI marketplace rates (as of 2024):

GPU

Hourly Rate

Daily Rate

4-Hour Session

RTX 3060

~$0.03

~$0.70

~$0.12

RTX 3090

~$0.06

~$1.50

~$0.25

RTX 4090

~$0.10

~$2.30

~$0.40

A100 40GB

~$0.17

~$4.00

~$0.70

A100 80GB

~$0.25

~$6.00

~$1.00

Prices vary by provider and demand. Check CLORE.AI Marketplace for current rates.

Save money:

Use Spot market for flexible workloads (often 30-50% cheaper)
Pay with CLORE tokens
Compare prices across different providers

PreviousKohya Training NextDeepSpeed Training

Last updated 11 days ago

Was this helpful?

hashtagRenting on CLORE.AI

hashtagAccess Your Server

hashtagWhat is LoRA/QLoRA?

hashtagRequirements

hashtagQuick Deploy

hashtagAccessing Your Service

hashtagDataset Preparation

hashtagChat Format (Recommended)

hashtagInstruction Format

hashtagAlpaca Format

hashtagQLoRA Fine-tuning Script

hashtagUsing Axolotl (Easier)

hashtagAxolotl Config Examples

hashtagChat Model

hashtagCode Model

hashtagMerging LoRA Weights

hashtagConvert to GGUF

hashtagMonitoring Training

hashtagWeights & Biases

hashtagTensorBoard

hashtagBest Practices

hashtagHyperparameters

hashtagDataset Size

hashtagAvoiding Overfitting

hashtagMulti-GPU Training

hashtagSaving & Exporting

hashtagTroubleshooting

hashtagOOM Errors

hashtagTraining Loss Not Decreasing

hashtagNaN Loss

hashtagCost Estimate

Renting on CLORE.AI

Access Your Server

What is LoRA/QLoRA?

Requirements

Quick Deploy

Accessing Your Service

Dataset Preparation

Chat Format (Recommended)

Instruction Format

Alpaca Format

QLoRA Fine-tuning Script

Using Axolotl (Easier)

Axolotl Config Examples

Chat Model

Code Model

Merging LoRA Weights

Convert to GGUF

Monitoring Training

Weights & Biases

TensorBoard

Best Practices

Hyperparameters

Dataset Size

Avoiding Overfitting

Multi-GPU Training

Saving & Exporting

Troubleshooting

OOM Errors

Training Loss Not Decreasing

NaN Loss

Cost Estimate