Stable Audio

Generate music and sound effects with Stable Audio on Clore.ai

Generate music and sound effects with Stability AI's Stable Audio on CLORE.AI GPUs.

All examples can be run on GPU servers rented through CLORE.AI Marketplace.

Why Stable Audio?

• High quality - 44.1kHz stereo audio generation

• Variable length - Generate up to 95 seconds

• Versatile - Music, sound effects, ambient sounds

• Text-to-audio - Describe what you want to hear

• Open weights - Stable Audio Open available

Model Variants

Model	Duration	Quality	VRAM	License
Stable Audio Open	47 sec	Good	8GB	Open
Stable Audio 2.0	3 min	Excellent	12GB	Commercial

Quick Deploy on CLORE.AI

Docker Image:

pytorch/pytorch:2.11.0-cuda12.8-cudnn9-devel

Ports:

22/tcp
7860/http

Command:

pip install stable-audio-tools gradio && \
python -c "
import gradio as gr
import torch
from stable_audio_tools import get_pretrained_model
from stable_audio_tools.inference.generation import generate_diffusion_cond
import soundfile as sf
import tempfile

model, model_config = get_pretrained_model('stabilityai/stable-audio-open-1.0')
model = model.to('cuda')

def generate(prompt, duration, steps, seed):
    conditioning = [{
        'prompt': prompt,
        'seconds_start': 0,
        'seconds_total': duration
    }]

    generator = torch.Generator('cuda').manual_seed(seed) if seed > 0 else None

    output = generate_diffusion_cond(
        model,
        conditioning=conditioning,
        steps=steps,
        cfg_scale=7,
        sample_size=model_config['sample_size'],
        sample_rate=model_config['sample_rate'],
        device='cuda',
        seed=seed if seed > 0 else None
    )

    audio = output[0].T.cpu().numpy()

    with tempfile.NamedTemporaryFile(suffix='.wav', delete=False) as f:
        sf.write(f.name, audio, model_config['sample_rate'])
        return f.name

gr.Interface(
    fn=generate,
    inputs=[
        gr.Textbox(label='Prompt'),
        gr.Slider(1, 47, value=10, label='Duration (sec)'),
        gr.Slider(10, 150, value=100, label='Steps'),
        gr.Number(value=-1, label='Seed')
    ],
    outputs=gr.Audio(label='Generated Audio'),
    title='Stable Audio Open'
).launch(server_name='0.0.0.0', server_port=7860)
"

Accessing Your Service

After deployment, find your http_pub URL in My Orders:

1. Go to My Orders page

2. Click on your order

3. Find the http_pub URL (e.g., abc123.clorecloud.net)

Use https://YOUR_HTTP_PUB_URL instead of localhost in examples below.

Hardware Requirements

Model	Minimum GPU	Recommended
Stable Audio Open	RTX 3070 8GB	RTX 3090 24GB
Stable Audio 2.0	RTX 3090 12GB	RTX 4090 24GB

Installation

pip install stable-audio-tools torch torchaudio

Basic Usage

Text to Music

import torch
import torchaudio
from stable_audio_tools import get_pretrained_model
from stable_audio_tools.inference.generation import generate_diffusion_cond

# Load model
model, model_config = get_pretrained_model("stabilityai/stable-audio-open-1.0")
model = model.to("cuda")

sample_rate = model_config["sample_rate"]
sample_size = model_config["sample_size"]

# Define what you want
conditioning = [{
    "prompt": "Upbeat electronic dance music with a catchy synth melody, 128 BPM",
    "seconds_start": 0,
    "seconds_total": 30
}]

# Generate
output = generate_diffusion_cond(
    model,
    conditioning=conditioning,
    steps=100,
    cfg_scale=7,
    sample_size=sample_size,
    sample_rate=sample_rate,
    device="cuda"
)

# Save
audio = output[0].T
torchaudio.save("music.wav", audio.cpu(), sample_rate)

Sound Effects

conditioning = [{
    "prompt": "Thunderstorm with heavy rain and distant thunder",
    "seconds_start": 0,
    "seconds_total": 20
}]

output = generate_diffusion_cond(
    model,
    conditioning=conditioning,
    steps=100,
    cfg_scale=7,
    sample_size=sample_size,
    sample_rate=sample_rate,
    device="cuda"
)

torchaudio.save("thunderstorm.wav", output[0].T.cpu(), sample_rate)

Ambient Sounds

conditioning = [{
    "prompt": "Peaceful forest ambience with birds singing and gentle wind",
    "seconds_start": 0,
    "seconds_total": 45
}]

output = generate_diffusion_cond(
    model,
    conditioning=conditioning,
    steps=100,
    cfg_scale=7,
    sample_size=sample_size,
    sample_rate=sample_rate,
    device="cuda"
)

torchaudio.save("forest.wav", output[0].T.cpu(), sample_rate)

Prompt Examples

Music Genres

prompts = {
    "electronic": "Energetic EDM track with deep bass, synth arpeggios, and a driving beat, 130 BPM",
    "jazz": "Smooth jazz piano trio with upright bass and brushed drums, relaxed tempo",
    "rock": "Heavy rock guitar riff with distortion, drums, and bass, powerful and energetic",
    "classical": "Orchestral piece with strings and woodwinds, dramatic and cinematic",
    "ambient": "Atmospheric ambient soundscape with pads and subtle textures, dreamy",
    "hiphop": "Lo-fi hip hop beat with vinyl crackle, mellow piano, and chill drums, 85 BPM"
}

Sound Effects

prompts = {
    "explosion": "Massive explosion with debris and fire, cinematic",
    "footsteps": "Footsteps on gravel, slow walking pace",
    "car": "Sports car engine revving and accelerating",
    "water": "Water splashing and dripping in a cave",
    "wind": "Strong wind howling through mountains",
    "fire": "Crackling campfire with wood popping"
}

Ambient/Background

prompts = {
    "cafe": "Coffee shop ambience with quiet chatter and espresso machine",
    "ocean": "Ocean waves on a sandy beach, seagulls in distance",
    "city": "Busy city street with traffic, horns, and pedestrians",
    "rain": "Gentle rain on window with occasional thunder",
    "space": "Sci-fi spaceship interior hum and beeps"
}

Advanced Options

Controlling Generation

output = generate_diffusion_cond(
    model,
    conditioning=conditioning,
    steps=150,              # More steps = better quality
    cfg_scale=7,            # Prompt adherence (5-10)
    sample_size=sample_size,
    sample_rate=sample_rate,
    device="cuda",
    seed=42                 # Reproducible results
)

Variable Length

# Short sound effect (5 seconds)
conditioning = [{
    "prompt": "Door creaking open slowly",
    "seconds_start": 0,
    "seconds_total": 5
}]

# Medium clip (30 seconds)
conditioning = [{
    "prompt": "Upbeat rock music",
    "seconds_start": 0,
    "seconds_total": 30
}]

# Maximum length (47 seconds for Open)
conditioning = [{
    "prompt": "Ambient electronic music, evolving textures",
    "seconds_start": 0,
    "seconds_total": 47
}]

Batch Generation

import os

prompts = [
    "Energetic drum and bass track",
    "Calm piano melody",
    "Sci-fi laser sound effects",
    "Rain on a tin roof"
]

output_dir = "./audio_output"
os.makedirs(output_dir, exist_ok=True)

for i, prompt in enumerate(prompts):
    conditioning = [{
        "prompt": prompt,
        "seconds_start": 0,
        "seconds_total": 15
    }]

    output = generate_diffusion_cond(
        model,
        conditioning=conditioning,
        steps=100,
        cfg_scale=7,
        sample_size=sample_size,
        sample_rate=sample_rate,
        device="cuda"
    )

    torchaudio.save(f"{output_dir}/audio_{i}.wav", output[0].T.cpu(), sample_rate)
    print(f"Generated: {prompt[:30]}...")

    torch.cuda.empty_cache()

Gradio Web Interface

import gradio as gr
import torch
import torchaudio
from stable_audio_tools import get_pretrained_model
from stable_audio_tools.inference.generation import generate_diffusion_cond
import tempfile

model, model_config = get_pretrained_model("stabilityai/stable-audio-open-1.0")
model = model.to("cuda")

sample_rate = model_config["sample_rate"]
sample_size = model_config["sample_size"]

def generate_audio(prompt, duration, steps, cfg_scale, seed):
    conditioning = [{
        "prompt": prompt,
        "seconds_start": 0,
        "seconds_total": duration
    }]

    generator_seed = seed if seed > 0 else None

    output = generate_diffusion_cond(
        model,
        conditioning=conditioning,
        steps=steps,
        cfg_scale=cfg_scale,
        sample_size=sample_size,
        sample_rate=sample_rate,
        device="cuda",
        seed=generator_seed
    )

    audio = output[0].T.cpu()

    with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as f:
        torchaudio.save(f.name, audio, sample_rate)
        return f.name

demo = gr.Interface(
    fn=generate_audio,
    inputs=[
        gr.Textbox(label="Prompt", placeholder="Describe the audio you want..."),
        gr.Slider(1, 47, value=15, step=1, label="Duration (seconds)"),
        gr.Slider(20, 200, value=100, step=10, label="Steps"),
        gr.Slider(1, 15, value=7, step=0.5, label="CFG Scale"),
        gr.Number(value=-1, label="Seed (-1 for random)")
    ],
    outputs=gr.Audio(label="Generated Audio", type="filepath"),
    title="Stable Audio Open - Text to Audio",
    description="Generate music and sound effects from text descriptions. Running on CLORE.AI.",
    examples=[
        ["Upbeat electronic dance music with synths, 128 BPM", 20, 100, 7, 42],
        ["Thunderstorm with heavy rain", 15, 100, 7, 123],
        ["Peaceful piano melody, emotional", 30, 100, 7, 456]
    ]
)

demo.launch(server_name="0.0.0.0", server_port=7860)

Performance

Duration	Steps	GPU	Time
10 sec	100	RTX 3090	~15s
10 sec	100	RTX 4090	~10s
30 sec	100	RTX 3090	~40s
30 sec	100	RTX 4090	~25s
47 sec	100	RTX 4090	~40s

Quality Tips

Better Music

# Include tempo and style
prompt = "Energetic rock music, electric guitar, drums, bass, 140 BPM, high energy"

# Be specific about instruments
prompt = "Solo acoustic guitar fingerpicking, folk style, warm and intimate"

# Describe mood
prompt = "Melancholic piano piece, minor key, slow tempo, emotional and sad"

Better Sound Effects

# Be specific
prompt = "Single gunshot from a rifle, outdoor, echo"

# Include environment
prompt = "Footsteps on wooden floor, indoor, slow pace, creaking"

# Describe texture
prompt = "Fire crackling, large bonfire, wood popping, sparks"

Cost Estimate

Typical CLORE.AI marketplace rates:

GPU	Hourly Rate	~30sec clips/hour
RTX 3060 12GB	~$0.03	~50
RTX 3090 24GB	~$0.06	~90
RTX 4090 24GB	~$0.10	~140
A100 40GB	~$0.17	~200

Prices vary. Check CLORE.AI Marketplace for current rates.

Troubleshooting

Out of Memory

# Reduce duration
conditioning = [{
    "prompt": prompt,
    "seconds_total": 15  # Instead of 47
}]

# Or enable CPU offload
model.enable_model_cpu_offload()

Poor Quality Output

• Increase steps (150-200)

• Adjust CFG scale (try 5-10)

• Be more specific in prompt

• Try different seeds

No Sound / Silence

• Check prompt is descriptive enough

• Avoid very abstract descriptions

• Try known-working prompts first

Audio Artifacts

• Increase steps

• Lower CFG scale

• Reduce duration

• Check for GPU thermal throttling

Stable Audio vs Others

Feature	Stable Audio	AudioCraft	Bark
Music	Excellent	Excellent	Poor
SFX	Great	Good	Poor
Speech	No	No	Yes
Duration	47s / 3min	30s	15s
Quality	44.1kHz	32kHz	24kHz
Open	Partial	Yes	Yes

Use Stable Audio when:

• High-quality music generation

• Sound effects for games/video

• Background music

• Ambient soundscapes

Next Steps

• AudioCraft - Meta's music generation

• Bark TTS - Voice synthesis

• Demucs - Audio separation

• Whisper - Transcription