Kohya Training

Train LoRA, Dreambooth, and full fine-tunes for Stable Diffusion using Kohya's trainer.

All examples can be run on GPU servers rented through CLORE.AI Marketplace.

Renting on CLORE.AI

1. Visit CLORE.AI Marketplace

2. Filter by GPU type, VRAM, and price

3. Choose On-Demand (fixed rate) or Spot (bid price)

4. Configure your order:

   • Select Docker image

   • Set ports (TCP for SSH, HTTP for web UIs)

   • Add environment variables if needed

   • Enter startup command

5. Select payment: CLORE, BTC, or USDT/USDC

6. Create order and wait for deployment

Access Your Server

• Find connection details in My Orders

• Web interfaces: Use the HTTP port URL

• SSH: ssh -p <port> root@<proxy-address>

What is Kohya?

Kohya_ss is a training toolkit for:

• LoRA - Lightweight adapters (most popular)

• Dreambooth - Subject/style training

• Full fine-tune - Complete model training

• LyCORIS - Advanced LoRA variants

Requirements

Training Type	Min VRAM	Recommended
LoRA SD 1.5	6GB	RTX 3060
LoRA SDXL	12GB	RTX 3090
Dreambooth SD 1.5	12GB	RTX 3090
Dreambooth SDXL	24GB	RTX 4090

Quick Deploy

Docker Image:

pytorch/pytorch:2.5.1-cuda12.4-cudnn9-devel

Ports:

22/tcp
7860/http

Command:

apt-get update && apt-get install -y git libgl1 libglib2.0-0 && \
cd /workspace && \
git clone https://github.com/bmaltais/kohya_ss.git && \
cd kohya_ss && \
pip install -r requirements.txt && \
pip install xformers && \
python kohya_gui.py --listen 0.0.0.0 --server_port 7860

Accessing Your Service

After deployment, find your http_pub URL in My Orders:

1. Go to My Orders page

2. Click on your order

3. Find the http_pub URL (e.g., abc123.clorecloud.net)

Use https://YOUR_HTTP_PUB_URL instead of localhost in examples below.

Using the Web UI

1. Access at http://<proxy>:<port>

2. Select training type (LoRA, Dreambooth, etc.)

3. Configure settings

4. Start training

Dataset Preparation

Folder Structure

/workspace/dataset/
├── 10_mysubject/           # Repeats_conceptname
│   ├── image1.png
│   ├── image1.txt          # Caption file
│   ├── image2.png
│   └── image2.txt
└── 10_regularization/      # Optional reg images
    ├── reg1.png
    └── reg1.txt

Image Requirements

• Resolution: 512x512 (SD 1.5) or 1024x1024 (SDXL)

• Format: PNG or JPG

• Quantity: 10-50 images for LoRA

• Quality: Clear, well-lit, varied angles

Caption Files

Create .txt file with same name as image:

myimage.txt:

a photo of sks person, professional portrait, studio lighting, high quality

Auto-Captioning

Use BLIP for automatic captions:

from transformers import BlipProcessor, BlipForConditionalGeneration
from PIL import Image
import os

processor = BlipProcessor.from_pretrained("Salesforce/blip-image-captioning-base")
model = BlipForConditionalGeneration.from_pretrained("Salesforce/blip-image-captioning-base").to("cuda")

for img_file in os.listdir("./images"):
    if img_file.endswith(('.png', '.jpg')):
        image = Image.open(f"./images/{img_file}")
        inputs = processor(image, return_tensors="pt").to("cuda")
        output = model.generate(**inputs, max_new_tokens=50)
        caption = processor.decode(output[0], skip_special_tokens=True)

        txt_file = img_file.rsplit('.', 1)[0] + '.txt'
        with open(f"./images/{txt_file}", 'w') as f:
            f.write(caption)

LoRA Training (SD 1.5)

Configuration

In Kohya UI:

Setting	Value
Model	runwayml/stable-diffusion-v1-5
Network Rank	32-128
Network Alpha	16-64
Learning Rate	1e-4
Batch Size	1-4
Epochs	10-20
Optimizer	AdamW8bit

Command Line Training

accelerate launch --num_cpu_threads_per_process=2 train_network.py \
    --pretrained_model_name_or_path="runwayml/stable-diffusion-v1-5" \
    --train_data_dir="/workspace/dataset" \
    --output_dir="/workspace/output" \
    --output_name="my_lora" \
    --resolution=512 \
    --train_batch_size=1 \
    --max_train_epochs=10 \
    --learning_rate=1e-4 \
    --network_module=networks.lora \
    --network_dim=32 \
    --network_alpha=16 \
    --mixed_precision=fp16 \
    --save_precision=fp16 \
    --optimizer_type=AdamW8bit \
    --lr_scheduler=cosine \
    --cache_latents \
    --xformers \
    --save_every_n_epochs=2

LoRA Training (SDXL)

accelerate launch train_network.py \
    --pretrained_model_name_or_path="stabilityai/stable-diffusion-xl-base-1.0" \
    --train_data_dir="/workspace/dataset" \
    --output_dir="/workspace/output" \
    --output_name="my_sdxl_lora" \
    --resolution=1024 \
    --train_batch_size=1 \
    --max_train_epochs=10 \
    --learning_rate=1e-4 \
    --network_module=networks.lora \
    --network_dim=32 \
    --network_alpha=16 \
    --mixed_precision=bf16 \
    --save_precision=fp16 \
    --optimizer_type=Adafactor \
    --cache_latents \
    --xformers \
    --save_every_n_epochs=2

Dreambooth Training

Subject Training

accelerate launch train_dreambooth.py \
    --pretrained_model_name_or_path="runwayml/stable-diffusion-v1-5" \
    --instance_data_dir="/workspace/dataset/instance" \
    --class_data_dir="/workspace/dataset/class" \
    --output_dir="/workspace/output" \
    --instance_prompt="a photo of sks person" \
    --class_prompt="a photo of person" \
    --with_prior_preservation \
    --prior_loss_weight=1.0 \
    --num_class_images=200 \
    --resolution=512 \
    --train_batch_size=1 \
    --learning_rate=2e-6 \
    --max_train_steps=1000 \
    --mixed_precision=fp16 \
    --gradient_checkpointing

Style Training

accelerate launch train_dreambooth.py \
    --pretrained_model_name_or_path="runwayml/stable-diffusion-v1-5" \
    --instance_data_dir="/workspace/dataset/style" \
    --output_dir="/workspace/output" \
    --instance_prompt="painting in the style of xyz" \
    --resolution=512 \
    --train_batch_size=1 \
    --learning_rate=5e-6 \
    --max_train_steps=2000 \
    --mixed_precision=fp16

Training Tips

Optimal Settings

Parameter	Person/Character	Style	Object
Network Rank	64-128	32-64	32
Network Alpha	32-64	16-32	16
Learning Rate	1e-4	5e-5	1e-4
Epochs	15-25	10-15	10-15

Avoiding Overfitting

• Use regularization images

• Lower learning rate

• Fewer epochs

• Increase network alpha

Avoiding Underfitting

• More training images

• Higher learning rate

• More epochs

• Lower network alpha

Monitoring Training

TensorBoard

tensorboard --logdir /workspace/output/logs --port 6006 --bind_all

Key Metrics

• loss - Should decrease then stabilize

• lr - Learning rate schedule

• epoch - Training progress

Testing Your LoRA

With Automatic1111

Copy LoRA to:

stable-diffusion-webui/models/Lora/my_lora.safetensors

Use in prompt:

<lora:my_lora:0.8> a photo of sks person

With ComfyUI

Load LoRA node and connect to model.

With Diffusers

from diffusers import StableDiffusionPipeline
import torch

pipe = StableDiffusionPipeline.from_pretrained(
    "runwayml/stable-diffusion-v1-5",
    torch_dtype=torch.float16
).to("cuda")

pipe.load_lora_weights("/workspace/output/my_lora.safetensors")

image = pipe("a photo of sks person, professional portrait").images[0]

Advanced Training

LyCORIS (LoHa, LoKR)

accelerate launch train_network.py \
    --network_module=lycoris.kohya \
    --network_args "algo=loha" "conv_dim=4" "conv_alpha=2" \
    ...

Textual Inversion

accelerate launch train_textual_inversion.py \
    --pretrained_model_name_or_path="runwayml/stable-diffusion-v1-5" \
    --train_data_dir="/workspace/dataset" \
    --learnable_property="style" \
    --placeholder_token="<my-style>" \
    --initializer_token="art" \
    --resolution=512 \
    --train_batch_size=1 \
    --max_train_steps=3000 \
    --learning_rate=5e-4

Saving & Exporting

Download Trained Model

scp -P <port> root@<proxy>:/workspace/output/my_lora.safetensors ./

Convert Formats

# SafeTensors to PyTorch
from safetensors.torch import load_file, save_file
import torch

state_dict = load_file("model.safetensors")
torch.save(state_dict, "model.pt")

Cost Estimate

Typical CLORE.AI marketplace rates (as of 2024):

GPU	Hourly Rate	Daily Rate	4-Hour Session
RTX 3060	~$0.03	~$0.70	~$0.12
RTX 3090	~$0.06	~$1.50	~$0.25
RTX 4090	~$0.10	~$2.30	~$0.40
A100 40GB	~$0.17	~$4.00	~$0.70
A100 80GB	~$0.25	~$6.00	~$1.00

Prices vary by provider and demand. Check CLORE.AI Marketplace for current rates.

Save money:

• Use Spot market for flexible workloads (often 30-50% cheaper)

• Pay with CLORE tokens

• Compare prices across different providers

Troubleshooting

OOM Error

• Reduce batch size to 1

• Enable gradient checkpointing

• Use 8bit optimizer

• Lower resolution

Poor Results

• More/better training images

• Adjust learning rate

• Check captions match images

• Try different network rank

Training Crashes

• Check CUDA version

• Update xformers

• Reduce batch size

• Check disk space