LivePortrait

Create realistic animated portraits from single images on Clore.ai

Create realistic animated portraits from single images.

All examples can be run on GPU servers rented through CLORE.AI Marketplace.

All examples in this guide can be run on GPU servers rented through CLORE.AI Marketplace marketplace.

Renting on CLORE.AI

1. Visit CLORE.AI Marketplace

2. Filter by GPU type, VRAM, and price

3. Choose On-Demand (fixed rate) or Spot (bid price)

4. Configure your order:

   • Select Docker image

   • Set ports (TCP for SSH, HTTP for web UIs)

   • Add environment variables if needed

   • Enter startup command

5. Select payment: CLORE, BTC, or USDT/USDC

6. Create order and wait for deployment

Access Your Server

• Find connection details in My Orders

• Web interfaces: Use the HTTP port URL

• SSH: ssh -p <port> root@<proxy-address>

What is LivePortrait?

LivePortrait by Kuaishou enables:

• Animate any portrait with driving video

• Single photo to video animation

• Expression and pose transfer

• Real-time capable inference

Resources

• GitHub: KwaiVGI/LivePortrait

• Paper: LivePortrait Paper

• HuggingFace: KwaiVGI/LivePortrait

• Demo: HuggingFace Space

Recommended Hardware

Component	Minimum	Recommended	Optimal
GPU	RTX 3070 8GB	RTX 4080 16GB	RTX 4090 24GB
VRAM	8GB	16GB	24GB
CPU	4 cores	8 cores	16 cores
RAM	16GB	32GB	64GB
Storage	30GB SSD	50GB NVMe	100GB NVMe
Internet	100 Mbps	500 Mbps	1 Gbps

Quick Deploy on CLORE.AI

Docker Image:

pytorch/pytorch:2.5.1-cuda12.4-cudnn9-devel

Ports:

22/tcp
7860/http

Command:

cd /workspace && \
git clone https://github.com/KwaiVGI/LivePortrait.git && \
cd LivePortrait && \
pip install -r requirements.txt && \
python app.py

Accessing Your Service

After deployment, find your http_pub URL in My Orders:

1. Go to My Orders page

2. Click on your order

3. Find the http_pub URL (e.g., abc123.clorecloud.net)

Use https://YOUR_HTTP_PUB_URL instead of localhost in examples below.

Installation

git clone https://github.com/KwaiVGI/LivePortrait.git
cd LivePortrait
pip install -r requirements.txt

# Download pretrained models
huggingface-cli download KwaiVGI/LivePortrait --local-dir pretrained_weights

What You Can Create

Virtual Avatars

• AI influencers and virtual anchors

• Customer service avatars

• Educational presenters

Content Creation

• Social media content

• Marketing materials

• Music video concepts

Entertainment

• Animate historical photos

• Character animations

• Interactive experiences

Professional Uses

• Video conferencing avatars

• Presentation assistants

• Training simulations

Basic Usage

Command Line

python inference.py \
    --source_image path/to/portrait.jpg \
    --driving_video path/to/driving.mp4 \
    --output_path output.mp4

Python API

from liveportrait import LivePortraitPipeline

# Initialize pipeline
pipeline = LivePortraitPipeline(
    device="cuda",
    model_path="./pretrained_weights"
)

# Animate portrait
result = pipeline.animate(
    source_image="portrait.jpg",
    driving_video="driving.mp4"
)

result.save("animated_portrait.mp4")

Portrait with Expression Control

from liveportrait import LivePortraitPipeline
import cv2

pipeline = LivePortraitPipeline(device="cuda")

# Control specific expressions
expressions = {
    "smile": 0.8,
    "eyebrow_raise": 0.3,
    "head_pitch": -5,  # degrees
    "head_yaw": 10
}

result = pipeline.animate_with_expression(
    source_image="portrait.jpg",
    expressions=expressions,
    num_frames=60,
    fps=30
)

result.save("expression_controlled.mp4")

Batch Processing

import os
from liveportrait import LivePortraitPipeline

pipeline = LivePortraitPipeline(device="cuda")

# Animate multiple portraits with same driving video
portraits = [
    "portrait1.jpg",
    "portrait2.jpg",
    "portrait3.jpg"
]

driving = "speech_driving.mp4"
output_dir = "./animated"
os.makedirs(output_dir, exist_ok=True)

for i, portrait in enumerate(portraits):
    print(f"Processing {i+1}/{len(portraits)}: {portrait}")

    result = pipeline.animate(
        source_image=portrait,
        driving_video=driving
    )

    result.save(f"{output_dir}/animated_{i:03d}.mp4")

Gradio Interface

import gradio as gr
from liveportrait import LivePortraitPipeline
import tempfile

pipeline = LivePortraitPipeline(device="cuda")

def animate(source_image, driving_video):
    with tempfile.NamedTemporaryFile(suffix=".mp4", delete=False) as f:
        result = pipeline.animate(
            source_image=source_image,
            driving_video=driving_video
        )
        result.save(f.name)
        return f.name

demo = gr.Interface(
    fn=animate,
    inputs=[
        gr.Image(type="filepath", label="Portrait Image"),
        gr.Video(label="Driving Video")
    ],
    outputs=gr.Video(label="Animated Portrait"),
    title="LivePortrait - Animate Any Portrait",
    description="Upload a portrait and a driving video to create an animated video. Running on CLORE.AI GPU servers."
)

demo.launch(server_name="0.0.0.0", server_port=7860)

Real-Time Webcam Animation

import cv2
from liveportrait import LivePortraitPipeline

pipeline = LivePortraitPipeline(device="cuda")

# Load source portrait
source = cv2.imread("portrait.jpg")
pipeline.set_source(source)

# Open webcam
cap = cv2.VideoCapture(0)

while True:
    ret, frame = cap.read()
    if not ret:
        break

    # Animate with current frame as driving
    animated = pipeline.animate_frame(frame)

    cv2.imshow("LivePortrait", animated)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

Integration with TTS

Create talking avatars with text-to-speech:

from liveportrait import LivePortraitPipeline
from TTS.api import TTS

# Generate speech
tts = TTS("tts_models/multilingual/multi-dataset/xtts_v2")
tts.tts_to_file(
    text="Hello! Welcome to our presentation.",
    file_path="speech.wav",
    speaker_wav="reference_voice.wav",
    language="en"
)

# Generate lip-sync driving video from audio

# (Using separate lip-sync tool or pre-made driving video)

# Animate portrait
pipeline = LivePortraitPipeline(device="cuda")
result = pipeline.animate(
    source_image="presenter.jpg",
    driving_video="lip_sync_driving.mp4"
)
result.save("talking_avatar.mp4")

Performance

Resolution	GPU	FPS	Latency
256x256	RTX 3070	30	33ms
256x256	RTX 4090	60+	16ms
512x512	RTX 4090	30	33ms
512x512	A100	45	22ms

Common Problems & Solutions

Face Not Detected

Problem: "No face detected in source image"

Solutions:

• Ensure face is clearly visible and front-facing

• Use good lighting in source image

• Crop image to focus on face

• Minimum face size: 128x128 pixels

Motion Doesn't Match

Problem: Animation doesn't follow driving video

Solutions:

• Use driving videos with clear facial movements

• Ensure driving video has similar face orientation

• Try different driving videos

Quality Issues

Problem: Output looks blurry or distorted

Solutions:

# Use higher quality settings
result = pipeline.animate(
    source_image=source,
    driving_video=driving,
    output_size=512,  # Higher resolution
    enhance_face=True  # Enable face enhancement
)

Real-Time Lag

Problem: Webcam animation is laggy

Solutions:

• Use smaller output resolution (256x256)

• Enable TensorRT optimization

• Use RTX 4090 or better for real-time

pipeline = LivePortraitPipeline(
    device="cuda",
    use_tensorrt=True  # Enable TensorRT
)

Audio Sync Issues

Problem: Lip movements don't match audio

Solutions:

• Use audio-to-driving video generation

• Adjust video timing in post-processing

• Use Wav2Lip for better lip sync

Troubleshooting

Face not detected

• Ensure face is clearly visible in source

• Use front-facing photos

• Check image resolution (512+ recommended)

Animation looks unnatural

• Source and driving video should have similar face angles

• Avoid extreme expressions in driving video

• Use shorter driving clips

Output video corrupted

• Install ffmpeg: apt install ffmpeg

• Check output format compatibility

• Ensure enough disk space

CUDA errors

• Install compatible PyTorch version

• Check CUDA version matches requirements

Cost Estimate

Typical CLORE.AI marketplace rates (as of 2024):

GPU	Hourly Rate	Daily Rate	4-Hour Session
RTX 3060	~$0.03	~$0.70	~$0.12
RTX 3090	~$0.06	~$1.50	~$0.25
RTX 4090	~$0.10	~$2.30	~$0.40
A100 40GB	~$0.17	~$4.00	~$0.70
A100 80GB	~$0.25	~$6.00	~$1.00

Prices vary by provider and demand. Check CLORE.AI Marketplace for current rates.

Save money:

• Use Spot market for flexible workloads (often 30-50% cheaper)

• Pay with CLORE tokens

• Compare prices across different providers

Next Steps

• SadTalker - Alternative talking head

• Wav2Lip - Better lip sync

• XTTS - Voice generation