OpenSora

Generate videos with OpenSora, the open-source Sora alternative, on CLORE.AI GPUs.

All examples can be run on GPU servers rented through CLORE.AI Marketplace.

Why OpenSora?

• Open source - Full Apache 2.0 license

• Sora-inspired - DiT architecture like OpenAI's Sora

• Scalable - Multiple model sizes and resolutions

• Long videos - Generate up to 16 seconds

• Active development - Regular updates and improvements

Model Variants

Model	Resolution	Duration	VRAM	Quality
OpenSora 1.2	720p	16s	24GB	Excellent
OpenSora 1.1	480p	8s	16GB	Good
OpenSora 1.0	256p	4s	8GB	Basic
Open-Sora-Plan	512p	10s	20GB	Great

Quick Deploy on CLORE.AI

Docker Image:

pytorch/pytorch:2.5.1-cuda12.4-cudnn9-devel

Ports:

22/tcp
7860/http

Command:

git clone https://github.com/hpcaitech/Open-Sora && \
cd Open-Sora && \
pip install -e . && \
pip install gradio && \
python scripts/inference.py \
    --prompt "A cat playing with yarn" \
    --num-frames 51 \
    --resolution 480p \
    --save-dir ./outputs

Accessing Your Service

After deployment, find your http_pub URL in My Orders:

1. Go to My Orders page

2. Click on your order

3. Find the http_pub URL (e.g., abc123.clorecloud.net)

Use https://YOUR_HTTP_PUB_URL instead of localhost in examples below.

Hardware Requirements

Model Version	Minimum GPU	Recommended	Optimal
OpenSora 1.0	RTX 3070 8GB	RTX 3090 24GB	RTX 4090
OpenSora 1.1	RTX 3090 16GB	RTX 4090 24GB	A100 40GB
OpenSora 1.2	RTX 4090 24GB	A100 40GB	A100 80GB

Installation

From Source

git clone https://github.com/hpcaitech/Open-Sora
cd Open-Sora

# Install dependencies
pip install -e .

# Download model weights
python scripts/download_weights.py --version 1.2

Using pip

pip install opensora

Basic Usage

Command Line

# Simple generation
python scripts/inference.py \
    --prompt "A beautiful sunset over the ocean, cinematic" \
    --num-frames 51 \
    --resolution 480p \
    --save-dir ./outputs

# High quality
python scripts/inference.py \
    --prompt "A majestic eagle soaring through clouds" \
    --num-frames 102 \
    --resolution 720p \
    --num-sampling-steps 100 \
    --save-dir ./outputs

Python API

import torch
from opensora.models import OpenSoraModel
from opensora.utils import export_to_video

# Load model
model = OpenSoraModel.from_pretrained("hpcaitech/OpenSora-v1.2")
model.to("cuda")

# Generate video
prompt = "A rocket launching into space, dramatic lighting, cinematic"

video = model.generate(
    prompt=prompt,
    num_frames=51,
    height=480,
    width=854,
    num_inference_steps=50,
    guidance_scale=7.0
)

# Save
export_to_video(video, "rocket.mp4", fps=24)

Advanced Generation

With Negative Prompts

video = model.generate(
    prompt="Professional photography of a tiger in the wild",
    negative_prompt="blurry, low quality, distorted, artifacts",
    num_frames=51,
    num_inference_steps=75,
    guidance_scale=7.5
)

Long Videos

# Generate 16 seconds at 24fps
video = model.generate(
    prompt="Time-lapse of flowers blooming in a garden",
    num_frames=384,  # 16 seconds at 24fps
    height=480,
    width=854,
    num_inference_steps=100
)

export_to_video(video, "timelapse.mp4", fps=24)

High Resolution

# 720p generation (requires more VRAM)
video = model.generate(
    prompt="Aerial view of a city at night with lights",
    num_frames=51,
    height=720,
    width=1280,
    num_inference_steps=75
)

Prompt Examples

Cinematic

prompts = [
    "Cinematic shot of a samurai drawing his sword, dramatic lighting, 4K",
    "Epic wide shot of a castle on a cliff during a storm",
    "Slow motion of water droplets falling into a still pond",
    "Tracking shot through a neon-lit cyberpunk alley at night"
]

Nature

prompts = [
    "Aurora borealis dancing over snowy mountains, time-lapse",
    "Macro shot of a butterfly emerging from cocoon",
    "Ocean waves crashing on volcanic rocks at sunset",
    "Fog rolling through an ancient forest at dawn"
]

Abstract

prompts = [
    "Colorful paint drops falling into water, slow motion",
    "Fractals evolving and transforming, psychedelic colors",
    "Liquid metal morphing into different shapes"
]

Configuration Options

Resolution Presets

resolutions = {
    "256p": (256, 455),
    "360p": (360, 640),
    "480p": (480, 854),
    "720p": (720, 1280),
    "1080p": (1080, 1920)  # High VRAM required
}

Quality Settings

# Fast preview
config_fast = {
    "num_frames": 25,
    "num_inference_steps": 25,
    "guidance_scale": 5.0
}

# Balanced
config_balanced = {
    "num_frames": 51,
    "num_inference_steps": 50,
    "guidance_scale": 7.0
}

# Maximum quality
config_quality = {
    "num_frames": 102,
    "num_inference_steps": 100,
    "guidance_scale": 7.5
}

Gradio Interface

import gradio as gr
import torch
from opensora.models import OpenSoraModel
from opensora.utils import export_to_video
import tempfile

model = OpenSoraModel.from_pretrained("hpcaitech/OpenSora-v1.2")
model.to("cuda")

def generate_video(prompt, negative_prompt, frames, steps, guidance, resolution, seed):
    res_map = {"480p": (480, 854), "720p": (720, 1280)}
    height, width = res_map.get(resolution, (480, 854))

    generator = torch.Generator("cuda").manual_seed(seed) if seed > 0 else None

    video = model.generate(
        prompt=prompt,
        negative_prompt=negative_prompt,
        num_frames=frames,
        height=height,
        width=width,
        num_inference_steps=steps,
        guidance_scale=guidance,
        generator=generator
    )

    with tempfile.NamedTemporaryFile(suffix=".mp4", delete=False) as f:
        export_to_video(video, f.name, fps=24)
        return f.name

demo = gr.Interface(
    fn=generate_video,
    inputs=[
        gr.Textbox(label="Prompt", lines=2),
        gr.Textbox(label="Negative Prompt", value="blurry, low quality"),
        gr.Slider(25, 200, value=51, step=1, label="Frames"),
        gr.Slider(20, 150, value=50, step=5, label="Steps"),
        gr.Slider(3, 15, value=7, step=0.5, label="Guidance"),
        gr.Dropdown(["480p", "720p"], value="480p", label="Resolution"),
        gr.Number(value=-1, label="Seed")
    ],
    outputs=gr.Video(label="Generated Video"),
    title="OpenSora - Text to Video",
    description="Generate videos using OpenSora. Running on CLORE.AI."
)

demo.launch(server_name="0.0.0.0", server_port=7860)

Memory Optimization

# Enable memory optimizations
model.enable_model_cpu_offload()
model.enable_vae_tiling()

# For very low VRAM
model.enable_sequential_cpu_offload()

# Use lower precision
model = OpenSoraModel.from_pretrained(
    "hpcaitech/OpenSora-v1.2",
    torch_dtype=torch.float16
)

Batch Generation

import os

prompts = [
    "A phoenix rising from flames",
    "Rain falling on city streets at night",
    "Flowers blooming time-lapse",
    "Northern lights over mountains"
]

output_dir = "./videos"
os.makedirs(output_dir, exist_ok=True)

for i, prompt in enumerate(prompts):
    print(f"Generating {i+1}/{len(prompts)}: {prompt[:40]}...")

    video = model.generate(
        prompt=prompt,
        num_frames=51,
        num_inference_steps=50
    )

    export_to_video(video, f"{output_dir}/video_{i:03d}.mp4", fps=24)

    # Clear memory between generations
    torch.cuda.empty_cache()

Performance

Resolution	Frames	Steps	GPU	Time
480p	51	50	RTX 4090	~3 min
480p	51	50	A100 40GB	~2 min
720p	51	50	A100 40GB	~5 min
720p	102	100	A100 80GB	~15 min

Cost Estimate

Typical CLORE.AI marketplace rates:

GPU	Hourly Rate	~51 frame 480p videos/hour
RTX 4090 24GB	~$0.10	~15-20
A100 40GB	~$0.17	~25-30
A100 80GB	~$0.25	~35 (can do 720p)

Prices vary. Check CLORE.AI Marketplace for current rates.

Troubleshooting

Out of Memory

# Use smaller resolution
python scripts/inference.py --resolution 360p --num-frames 25

# Enable CPU offload
python scripts/inference.py --cpu-offload

# Reduce batch size
python scripts/inference.py --batch-size 1

Slow Generation

• Reduce num_inference_steps (30-50 often enough)

• Use lower resolution for previews

• Ensure GPU is being utilized (check nvidia-smi)

Poor Quality

• Increase steps to 75-100

• Use more descriptive prompts

• Add negative prompts for artifacts

• Try different guidance scales (5-10)

Video Artifacts

• Lower guidance scale

• Increase inference steps

• Use temporal smoothing

• Post-process with video stabilization

OpenSora vs Others

Feature	OpenSora 1.2	Hunyuan	Wan2.1	SVD
Architecture	DiT	DiT	DiT	U-Net
Max Duration	16s	5s	5s	4s
Max Resolution	720p	720p	720p	576p
Quality	Great	Excellent	Excellent	Good
Speed	Medium	Slow	Fast	Fast
License	Apache 2.0	Open	Apache 2.0	Open

Use OpenSora when:

• Need longer video generation

• Want full Apache 2.0 license

• Interested in Sora-like architecture

• Need active community support

Open-Sora-Plan Alternative

Another open-source option:

git clone https://github.com/PKU-YuanGroup/Open-Sora-Plan
cd Open-Sora-Plan
pip install -e .

python scripts/inference.py \
    --prompt "Your prompt here" \
    --output video.mp4

Next Steps

• Hunyuan Video - High quality T2V

• Wan2.1 Video - Fast generation

• Stable Video Diffusion - Image animation

• RIFE Interpolation - Frame interpolation