LTX-2 (Audio + Video)

Generate videos with native audio — foley, ambience, and lip-sync — using LTX-2 on Clore.ai GPUs.

LTX-2 (January 2026) is Lightricks' second-generation video foundation model and the first open-weight model to produce synchronized audio alongside video in a single forward pass. At 19B parameters it generates clips with foley sound effects, ambient audio, and lip-synced speech without requiring a separate audio model. The architecture builds on the original LTX-Video's speed advantage while dramatically expanding capability.

Renting a GPU on Clore.ai is the most practical way to run a 19B-parameter model — no $2,000 GPU purchase required, just spin up a machine and start generating.

Key Features

• Native audio generation — foley effects, environmental ambience, and lip-synced dialogue produced jointly with video frames.

• 19B parameters — significantly larger transformer backbone than LTX-Video v1, delivering sharper detail and more coherent motion.

• Text-to-Video + Image-to-Video — both modalities supported with audio output.

• Up to 720p resolution — higher fidelity output than the v1 model.

• Joint audio-visual latent space — a unified VAE encodes both video and audio, keeping them temporally aligned.

• Open weights — released under a permissive license for commercial use.

• Diffusers integration — compatible with the Hugging Face diffusers ecosystem.

Requirements

Component	Minimum	Recommended
GPU VRAM	16 GB (with offloading)	24+ GB
System RAM	32 GB	64 GB
Disk	50 GB	80 GB
Python	3.10+	3.11
CUDA	12.1+	12.4
diffusers	0.33+	latest

Clore.ai GPU recommendation: An RTX 4090 (24 GB, ~$0.5–2/day) is the minimum for comfortable 720p generation with audio. For batch workloads or faster iteration, filter for dual-4090 or A6000 (48 GB) listings on the Clore.ai marketplace.

Quick Start

# Install dependencies
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124
pip install diffusers transformers accelerate sentencepiece
pip install imageio[ffmpeg] soundfile scipy

# Verify GPU
python -c "import torch; print(torch.cuda.get_device_name(0), torch.cuda.get_device_properties(0).total_mem // 1024**3, 'GB')"

Usage Examples

Text-to-Video with Audio

import torch
from diffusers import LTXPipeline
from diffusers.utils import export_to_video
import soundfile as sf

# Load LTX-2 (ensure you have the correct model ID when released)
pipe = LTXPipeline.from_pretrained(
    "Lightricks/LTX-Video-2",
    torch_dtype=torch.bfloat16,
)
pipe.to("cuda")
pipe.enable_model_cpu_offload()

prompt = (
    "A blacksmith hammering glowing metal on an anvil, sparks flying, "
    "rhythmic clanging of hammer on steel, ambient workshop noise"
)

output = pipe(
    prompt=prompt,
    negative_prompt="silent, blurry, low quality",
    num_frames=121,
    width=1280,
    height=720,
    num_inference_steps=40,
    guidance_scale=7.0,
    generator=torch.Generator("cuda").manual_seed(42),
)

# Export video frames
export_to_video(output.frames[0], "blacksmith.mp4", fps=24)

# Export audio if available
if hasattr(output, "audio") and output.audio is not None:
    sf.write("blacksmith_audio.wav", output.audio, samplerate=16000)
    print("Audio saved separately — mux with ffmpeg:")
    print("  ffmpeg -i blacksmith.mp4 -i blacksmith_audio.wav -c:v copy -c:a aac output.mp4")

print("Done: blacksmith.mp4")

Image-to-Video with Lip-Sync Audio

import torch
from PIL import Image
from diffusers import LTXImageToVideoPipeline
from diffusers.utils import export_to_video

pipe = LTXImageToVideoPipeline.from_pretrained(
    "Lightricks/LTX-Video-2",
    torch_dtype=torch.bfloat16,
)
pipe.to("cuda")
pipe.enable_model_cpu_offload()

# Portrait image for lip-sync
image = Image.open("portrait.png").resize((720, 1280))

output = pipe(
    prompt="A person saying 'Welcome to the future of AI video' with clear enunciation, neutral background",
    image=image,
    num_frames=121,
    num_inference_steps=40,
    guidance_scale=7.0,
)

export_to_video(output.frames[0], "talking_head.mp4", fps=24)

Ambient Scene with Foley

import torch
from diffusers import LTXPipeline
from diffusers.utils import export_to_video

pipe = LTXPipeline.from_pretrained(
    "Lightricks/LTX-Video-2", torch_dtype=torch.bfloat16
).to("cuda")

# Audio-rich prompt — describe sounds explicitly
prompt = (
    "Rain falling on a tin roof in a tropical village, "
    "thunder rumbling in the distance, birds briefly chirping between rolls, "
    "puddles rippling on a dirt path"
)

output = pipe(
    prompt=prompt,
    num_frames=121,
    width=1280,
    height=720,
    num_inference_steps=40,
    guidance_scale=6.5,
)

export_to_video(output.frames[0], "rain_scene.mp4", fps=24)

Tips for Clore.ai Users

1. Describe sounds explicitly — LTX-2's audio branch responds to audio cues in the prompt. "Crackling fire", "footsteps on gravel", "crowd murmuring" yield better foley than vague descriptions.

2. CPU offloading is essential — at 19B parameters, the model needs enable_model_cpu_offload() on 24 GB cards. Budget 64 GB system RAM.

3. Persistent storage — the model checkpoint is ~40 GB. Mount a Clore.ai persistent volume and set HF_HOME to avoid re-downloading on every container restart.

4. Mux audio + video — if the pipeline outputs audio separately, combine with: ffmpeg -i video.mp4 -i audio.wav -c:v copy -c:a aac final.mp4.

5. bf16 only — the 19B model was trained in bf16; fp16 will cause numerical instability.

6. Batch in tmux — always run inside tmux on Clore.ai rentals to survive SSH disconnects.

7. Check model ID — as LTX-2 is freshly released (Jan 2026), verify the exact HuggingFace model ID on the Lightricks HF page before running.

Troubleshooting

Problem	Fix
OutOfMemoryError	Enable pipe.enable_model_cpu_offload(); ensure ≥64 GB system RAM
No audio in output	Audio generation may require explicit flag or updated diffusers; check model card for latest API
Audio/video desync	Re-mux with ffmpeg: ffmpeg -i video.mp4 -i audio.wav -c:v copy -c:a aac -shortest out.mp4
Very slow generation	19B model is compute-heavy; ~2–4 min per 5-sec clip on RTX 4090 is expected
NaN outputs	Use torch.bfloat16 — fp16 is not supported for this model scale
Disk space error	Model is ~40 GB; ensure ≥80 GB free disk before downloading
ModuleNotFoundError: soundfile	pip install soundfile — needed for WAV audio export