# OpenSora

{% hint style="info" %}
**Newer alternatives available!** [**FramePack**](https://docs.clore.ai/guides/video-generation/framepack) generates video with just 6GB VRAM, [**Wan2.1**](https://docs.clore.ai/guides/video-generation/wan-video) offers superior quality, and [**LTX-2**](https://docs.clore.ai/guides/video-generation/ltx-video-2) adds native audio generation.
{% endhint %}

Generate videos with OpenSora, the open-source Sora alternative, on CLORE.AI GPUs.

{% hint style="success" %}
All examples can be run on GPU servers rented through [CLORE.AI Marketplace](https://clore.ai/marketplace).
{% endhint %}

## Why OpenSora?

* **Open source** - Full Apache 2.0 license
* **Sora-inspired** - DiT architecture like OpenAI's Sora
* **Scalable** - Multiple model sizes and resolutions
* **Long videos** - Generate up to 16 seconds
* **Active development** - Regular updates and improvements

## Model Variants

| Model          | Resolution | Duration | VRAM | Quality   |
| -------------- | ---------- | -------- | ---- | --------- |
| OpenSora 1.2   | 720p       | 16s      | 24GB | Excellent |
| OpenSora 1.1   | 480p       | 8s       | 16GB | Good      |
| OpenSora 1.0   | 256p       | 4s       | 8GB  | Basic     |
| Open-Sora-Plan | 512p       | 10s      | 20GB | Great     |

## Quick Deploy on CLORE.AI

**Docker Image:**

```
pytorch/pytorch:2.5.1-cuda12.4-cudnn9-devel
```

**Ports:**

```
22/tcp
7860/http
```

**Command:**

```bash
git clone https://github.com/hpcaitech/Open-Sora && \
cd Open-Sora && \
pip install -e . && \
pip install gradio && \
python scripts/inference.py \
    --prompt "A cat playing with yarn" \
    --num-frames 51 \
    --resolution 480p \
    --save-dir ./outputs
```

## Accessing Your Service

After deployment, find your `http_pub` URL in **My Orders**:

1. Go to **My Orders** page
2. Click on your order
3. Find the `http_pub` URL (e.g., `abc123.clorecloud.net`)

Use `https://YOUR_HTTP_PUB_URL` instead of `localhost` in examples below.

## Hardware Requirements

| Model Version | Minimum GPU   | Recommended   | Optimal   |
| ------------- | ------------- | ------------- | --------- |
| OpenSora 1.0  | RTX 3070 8GB  | RTX 3090 24GB | RTX 4090  |
| OpenSora 1.1  | RTX 3090 16GB | RTX 4090 24GB | A100 40GB |
| OpenSora 1.2  | RTX 4090 24GB | A100 40GB     | A100 80GB |

## Installation

### From Source

```bash
git clone https://github.com/hpcaitech/Open-Sora
cd Open-Sora

# Install dependencies
pip install -e .

# Download model weights
python scripts/download_weights.py --version 1.2
```

### Using pip

```bash
pip install opensora
```

## Basic Usage

### Command Line

```bash
# Simple generation
python scripts/inference.py \
    --prompt "A beautiful sunset over the ocean, cinematic" \
    --num-frames 51 \
    --resolution 480p \
    --save-dir ./outputs

# High quality
python scripts/inference.py \
    --prompt "A majestic eagle soaring through clouds" \
    --num-frames 102 \
    --resolution 720p \
    --num-sampling-steps 100 \
    --save-dir ./outputs
```

### Python API

```python
import torch
from opensora.models import OpenSoraModel
from opensora.utils import export_to_video

# Load model
model = OpenSoraModel.from_pretrained("hpcaitech/OpenSora-v1.2")
model.to("cuda")

# Generate video
prompt = "A rocket launching into space, dramatic lighting, cinematic"

video = model.generate(
    prompt=prompt,
    num_frames=51,
    height=480,
    width=854,
    num_inference_steps=50,
    guidance_scale=7.0
)

# Save
export_to_video(video, "rocket.mp4", fps=24)
```

## Advanced Generation

### With Negative Prompts

```python
video = model.generate(
    prompt="Professional photography of a tiger in the wild",
    negative_prompt="blurry, low quality, distorted, artifacts",
    num_frames=51,
    num_inference_steps=75,
    guidance_scale=7.5
)
```

### Long Videos

```python
# Generate 16 seconds at 24fps
video = model.generate(
    prompt="Time-lapse of flowers blooming in a garden",
    num_frames=384,  # 16 seconds at 24fps
    height=480,
    width=854,
    num_inference_steps=100
)

export_to_video(video, "timelapse.mp4", fps=24)
```

### High Resolution

```python
# 720p generation (requires more VRAM)
video = model.generate(
    prompt="Aerial view of a city at night with lights",
    num_frames=51,
    height=720,
    width=1280,
    num_inference_steps=75
)
```

## Prompt Examples

### Cinematic

```python
prompts = [
    "Cinematic shot of a samurai drawing his sword, dramatic lighting, 4K",
    "Epic wide shot of a castle on a cliff during a storm",
    "Slow motion of water droplets falling into a still pond",
    "Tracking shot through a neon-lit cyberpunk alley at night"
]
```

### Nature

```python
prompts = [
    "Aurora borealis dancing over snowy mountains, time-lapse",
    "Macro shot of a butterfly emerging from cocoon",
    "Ocean waves crashing on volcanic rocks at sunset",
    "Fog rolling through an ancient forest at dawn"
]
```

### Abstract

```python
prompts = [
    "Colorful paint drops falling into water, slow motion",
    "Fractals evolving and transforming, psychedelic colors",
    "Liquid metal morphing into different shapes"
]
```

## Configuration Options

### Resolution Presets

```python
resolutions = {
    "256p": (256, 455),
    "360p": (360, 640),
    "480p": (480, 854),
    "720p": (720, 1280),
    "1080p": (1080, 1920)  # High VRAM required
}
```

### Quality Settings

```python
# Fast preview
config_fast = {
    "num_frames": 25,
    "num_inference_steps": 25,
    "guidance_scale": 5.0
}

# Balanced
config_balanced = {
    "num_frames": 51,
    "num_inference_steps": 50,
    "guidance_scale": 7.0
}

# Maximum quality
config_quality = {
    "num_frames": 102,
    "num_inference_steps": 100,
    "guidance_scale": 7.5
}
```

## Gradio Interface

```python
import gradio as gr
import torch
from opensora.models import OpenSoraModel
from opensora.utils import export_to_video
import tempfile

model = OpenSoraModel.from_pretrained("hpcaitech/OpenSora-v1.2")
model.to("cuda")

def generate_video(prompt, negative_prompt, frames, steps, guidance, resolution, seed):
    res_map = {"480p": (480, 854), "720p": (720, 1280)}
    height, width = res_map.get(resolution, (480, 854))

    generator = torch.Generator("cuda").manual_seed(seed) if seed > 0 else None

    video = model.generate(
        prompt=prompt,
        negative_prompt=negative_prompt,
        num_frames=frames,
        height=height,
        width=width,
        num_inference_steps=steps,
        guidance_scale=guidance,
        generator=generator
    )

    with tempfile.NamedTemporaryFile(suffix=".mp4", delete=False) as f:
        export_to_video(video, f.name, fps=24)
        return f.name

demo = gr.Interface(
    fn=generate_video,
    inputs=[
        gr.Textbox(label="Prompt", lines=2),
        gr.Textbox(label="Negative Prompt", value="blurry, low quality"),
        gr.Slider(25, 200, value=51, step=1, label="Frames"),
        gr.Slider(20, 150, value=50, step=5, label="Steps"),
        gr.Slider(3, 15, value=7, step=0.5, label="Guidance"),
        gr.Dropdown(["480p", "720p"], value="480p", label="Resolution"),
        gr.Number(value=-1, label="Seed")
    ],
    outputs=gr.Video(label="Generated Video"),
    title="OpenSora - Text to Video",
    description="Generate videos using OpenSora. Running on CLORE.AI."
)

demo.launch(server_name="0.0.0.0", server_port=7860)
```

## Memory Optimization

```python
# Enable memory optimizations
model.enable_model_cpu_offload()
model.enable_vae_tiling()

# For very low VRAM
model.enable_sequential_cpu_offload()

# Use lower precision
model = OpenSoraModel.from_pretrained(
    "hpcaitech/OpenSora-v1.2",
    torch_dtype=torch.float16
)
```

## Batch Generation

```python
import os

prompts = [
    "A phoenix rising from flames",
    "Rain falling on city streets at night",
    "Flowers blooming time-lapse",
    "Northern lights over mountains"
]

output_dir = "./videos"
os.makedirs(output_dir, exist_ok=True)

for i, prompt in enumerate(prompts):
    print(f"Generating {i+1}/{len(prompts)}: {prompt[:40]}...")

    video = model.generate(
        prompt=prompt,
        num_frames=51,
        num_inference_steps=50
    )

    export_to_video(video, f"{output_dir}/video_{i:03d}.mp4", fps=24)

    # Clear memory between generations
    torch.cuda.empty_cache()
```

## Performance

| Resolution | Frames | Steps | GPU       | Time     |
| ---------- | ------ | ----- | --------- | -------- |
| 480p       | 51     | 50    | RTX 4090  | \~3 min  |
| 480p       | 51     | 50    | A100 40GB | \~2 min  |
| 720p       | 51     | 50    | A100 40GB | \~5 min  |
| 720p       | 102    | 100   | A100 80GB | \~15 min |

## Cost Estimate

Typical CLORE.AI marketplace rates:

| GPU           | Hourly Rate | \~51 frame 480p videos/hour |
| ------------- | ----------- | --------------------------- |
| RTX 4090 24GB | \~$0.10     | \~15-20                     |
| A100 40GB     | \~$0.17     | \~25-30                     |
| A100 80GB     | \~$0.25     | \~35 (can do 720p)          |

*Prices vary. Check* [*CLORE.AI Marketplace*](https://clore.ai/marketplace) *for current rates.*

## Troubleshooting

### Out of Memory

```bash
# Use smaller resolution
python scripts/inference.py --resolution 360p --num-frames 25

# Enable CPU offload
python scripts/inference.py --cpu-offload

# Reduce batch size
python scripts/inference.py --batch-size 1
```

### Slow Generation

* Reduce `num_inference_steps` (30-50 often enough)
* Use lower resolution for previews
* Ensure GPU is being utilized (check `nvidia-smi`)

### Poor Quality

* Increase steps to 75-100
* Use more descriptive prompts
* Add negative prompts for artifacts
* Try different guidance scales (5-10)

### Video Artifacts

* Lower guidance scale
* Increase inference steps
* Use temporal smoothing
* Post-process with video stabilization

## OpenSora vs Others

| Feature        | OpenSora 1.2 | Hunyuan   | Wan2.1     | SVD   |
| -------------- | ------------ | --------- | ---------- | ----- |
| Architecture   | DiT          | DiT       | DiT        | U-Net |
| Max Duration   | 16s          | 5s        | 5s         | 4s    |
| Max Resolution | 720p         | 720p      | 720p       | 576p  |
| Quality        | Great        | Excellent | Excellent  | Good  |
| Speed          | Medium       | Slow      | Fast       | Fast  |
| License        | Apache 2.0   | Open      | Apache 2.0 | Open  |

**Use OpenSora when:**

* Need longer video generation
* Want full Apache 2.0 license
* Interested in Sora-like architecture
* Need active community support

## Open-Sora-Plan Alternative

Another open-source option:

```bash
git clone https://github.com/PKU-YuanGroup/Open-Sora-Plan
cd Open-Sora-Plan
pip install -e .

python scripts/inference.py \
    --prompt "Your prompt here" \
    --output video.mp4
```

## Next Steps

* [Hunyuan Video](https://docs.clore.ai/guides/video-generation/hunyuan-video) - High quality T2V
* [Wan2.1 Video](https://docs.clore.ai/guides/video-generation/wan-video) - Fast generation
* [Stable Video Diffusion](https://docs.clore.ai/guides/video-generation/stable-video-diffusion) - Image animation
* [RIFE Interpolation](https://docs.clore.ai/guides/video-processing/rife-interpolation) - Frame interpolation