> For the complete documentation index, see [llms.txt](https://docs.clore.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.clore.ai/guides/guides_v2-zh/talking-heads/wav2lip.md).

# Wav2Lip

在 Clore.ai GPU 上使用 Wav2Lip 将口型精确同步到任何音频

使用 Wav2Lip 将嘴唇与任意音频同步。

{% hint style="success" %}
所有示例都可以在通过以下方式租用的 GPU 服务器上运行 [CLORE.AI 市场](https://clore.ai/marketplace).
{% endhint %}

## 在 CLORE.AI 上租用

1. 访问 [CLORE.AI 市场](https://clore.ai/marketplace)
2. 按 GPU 类型、VRAM 和价格筛选
3. 选择 **按需** （固定费率）或 **竞价** （出价）
4. 配置你的订单：
   * 选择 Docker 镜像
   * 设置端口（SSH 用 TCP，Web UI 用 HTTP）
   * 如有需要，添加环境变量
   * 输入启动命令
5. 选择支付方式： **CLORE**, **BTC**，或 **USDT/USDC**
6. 创建订单并等待部署

### 访问你的服务器

* 在以下位置查找连接信息 **我的订单**
* Web 界面：使用 HTTP 端口 URL
* SSH： `ssh -p <port> root@<proxy-address>`

## 什么是 Wav2Lip？

Wav2Lip 提供：

* 适用于任何人脸的精准口型同步
* 适用于任何音频
* 视频或图像输入
* 支持实时

## 需求

| 模式  | 显存  | 推荐       |
| --- | --- | -------- |
| 基础  | 4GB | RTX 3060 |
| 高质量 | 6GB | RTX 3080 |
| 高清  | 8GB | RTX 4080 |

## 快速部署

**Docker 镜像：**

```
pytorch/pytorch:2.11.0-cuda12.8-cudnn9-devel
```

**端口：**

```
22/tcp
7860/http
```

**命令：**

```bash
cd /workspace && \
git clone https://github.com/Rudrabha/Wav2Lip.git && \
cd Wav2Lip && \
pip install -r requirements.txt && \
wget "https://huggingface.co/spaces/wav2lip/wav2lip/resolve/main/checkpoints/wav2lip_gan.pth" -P checkpoints/ && \
python app.py
```

## 访问你的服务

部署后，找到你的 `http_pub` URL 在 **我的订单**:

1. 前往 **我的订单** 页面
2. 点击你的订单
3. 找到 `http_pub` URL（例如， `abc123.clorecloud.net`)

使用 `https://YOUR_HTTP_PUB_URL` 替代 `localhost` 在下面的示例中。

## 安装

```bash
git clone https://github.com/Rudrabha/Wav2Lip.git
cd Wav2Lip
pip install -r requirements.txt

# 下载预训练模型
mkdir -p checkpoints
wget "https://huggingface.co/spaces/wav2lip/wav2lip/resolve/main/checkpoints/wav2lip.pth" -P checkpoints/
wget "https://huggingface.co/spaces/wav2lip/wav2lip/resolve/main/checkpoints/wav2lip_gan.pth" -P checkpoints/
```

## 基础用法

### 命令行

```bash
python inference.py \
    --checkpoint_path checkpoints/wav2lip_gan.pth \
    --face input_video.mp4 \
    --audio audio.wav \
    --outfile output.mp4
```

### 使用图像输入

```bash
python inference.py \
    --checkpoint_path checkpoints/wav2lip_gan.pth \
    --face face_image.jpg \
    --audio speech.wav \
    --outfile talking.mp4
```

## Python API

```python
import subprocess

def wav2lip_sync(face_path, audio_path, output_path, checkpoint="checkpoints/wav2lip_gan.pth"):
    cmd = [
        "python", "inference.py",
        "--checkpoint_path", checkpoint,
        "--face", face_path,
        "--audio", audio_path,
        "--outfile", output_path
    ]
    subprocess.run(cmd, check=True)
    return output_path

# 用法
result = wav2lip_sync(
    face_path="video.mp4",
    audio_path="new_audio.wav",
    output_path="synced.mp4"
)
```

## 质量选项

### 标准质量（更快）

```bash
python inference.py \
    --checkpoint_path checkpoints/wav2lip.pth \
    --face input.mp4 \
    --audio audio.wav \
    --outfile output.mp4
```

### 高质量（GAN）

```bash
python inference.py \
    --checkpoint_path checkpoints/wav2lip_gan.pth \
    --face input.mp4 \
    --audio audio.wav \
    --outfile output.mp4 \
    --pads 0 10 0 0 \
    --resize_factor 1
```

## 参数

```bash
python inference.py \
    --checkpoint_path checkpoints/wav2lip_gan.pth \
    --face video.mp4 \
    --audio audio.wav \
    --outfile result.mp4 \
    --pads 0 10 0 0 \      # 填充：上 右 下 左
    --resize_factor 1 \    # 缩小因子
    --crop "0 -1 0 -1" \   # 裁剪区域
    --box "-1 -1 -1 -1" \  # 人脸框（自动检测）
    --nosmooth            # 禁用时间平滑
```

### 填充提示

| 人脸位置 | 推荐填充值    |
| ---- | -------- |
| 居中   | 0 10 0 0 |
| 特写   | 0 15 0 0 |
| 远景   | 0 5 0 0  |

## 批量处理

```python
import os
import subprocess

def batch_wav2lip(faces_dir, audio_path, output_dir):
    os.makedirs(output_dir, exist_ok=True)

    for filename in os.listdir(faces_dir):
        if filename.endswith(('.mp4', '.jpg', '.png')):
            face_path = os.path.join(faces_dir, filename)
            output_path = os.path.join(output_dir, f"synced_{filename}")

            if filename.endswith(('.jpg', '.png')):
                output_path = output_path.rsplit('.', 1)[0] + '.mp4'

            cmd = [
                "python", "inference.py",
                "--checkpoint_path", "checkpoints/wav2lip_gan.pth",
                "--face", face_path,
                "--audio", audio_path,
                "--outfile", output_path
            ]

            try:
                subprocess.run(cmd, check=True)
                print(f"已处理：{filename}")
            except subprocess.CalledProcessError as e:
                print(f"处理 {filename} 时出错：{e}")

# 用法
batch_wav2lip("./faces", "speech.wav", "./outputs")
```

## Gradio 界面

```python
import gradio as gr
import subprocess
import tempfile
import os

def lip_sync(face_video, audio, quality):
    checkpoint = "checkpoints/wav2lip_gan.pth" if quality == "High (GAN)" else "checkpoints/wav2lip.pth"

    with tempfile.NamedTemporaryFile(suffix=".mp4", delete=False) as out_file:
        output_path = out_file.name

    cmd = [
        "python", "inference.py",
        "--checkpoint_path", checkpoint,
        "--face", face_video,
        "--audio", audio,
        "--outfile", output_path
    ]

    subprocess.run(cmd, check=True)
    return output_path

demo = gr.Interface(
    fn=lip_sync,
    inputs=[
        gr.Video(label="人脸视频/图像"),
        gr.Audio(type="filepath", label="音频"),
        gr.Radio(["Standard", "High (GAN)"], value="High (GAN)", label="质量")
    ],
    outputs=gr.Video(label="口型同步后的视频"),
    title="Wav2Lip - 口型同步"
)

demo.launch(server_name="0.0.0.0", server_port=7860)
```

## API 服务器

```python
from fastapi import FastAPI, UploadFile, File
from fastapi.responses import FileResponse
import tempfile
import subprocess
import os

app = FastAPI()

@app.post("/sync")
async def sync_lips(
    face: UploadFile = File(...),
    audio: UploadFile = File(...),
    quality: str = "gan"
):
    with tempfile.TemporaryDirectory() as tmpdir:
        # 保存上传文件
        face_ext = os.path.splitext(face.filename)[1]
        face_path = os.path.join(tmpdir, f"face{face_ext}")
        audio_path = os.path.join(tmpdir, "audio.wav")
        output_path = os.path.join(tmpdir, "output.mp4")

        with open(face_path, "wb") as f:
            f.write(await face.read())
        with open(audio_path, "wb") as f:
            f.write(await audio.read())

        # 运行 Wav2Lip
        checkpoint = "checkpoints/wav2lip_gan.pth" if quality == "gan" else "checkpoints/wav2lip.pth"

        cmd = [
            "python", "inference.py",
            "--checkpoint_path", checkpoint,
            "--face", face_path,
            "--audio", audio_path,
            "--outfile", output_path
        ]

        subprocess.run(cmd, check=True)

        return FileResponse(output_path, media_type="video/mp4")

# 运行：uvicorn server:app --host 0.0.0.0 --port 8000
```

## TTS + Wav2Lip 流程

完整的文本到视频：

```python
from TTS.api import TTS
import subprocess

def text_to_lipsync(text, face_path, output_path, language="en"):
    # 生成语音
    tts = TTS("tts_models/multilingual/multi-dataset/xtts_v2")
    audio_path = "temp_speech.wav"
    tts.tts_to_file(text=text, file_path=audio_path, language=language)

    # 口型同步
    cmd = [
        "python", "inference.py",
        "--checkpoint_path", "checkpoints/wav2lip_gan.pth",
        "--face", face_path,
        "--audio", audio_path,
        "--outfile", output_path
    ]
    subprocess.run(cmd, check=True)

    return output_path

# 用法
text_to_lipsync(
    "Hello, welcome to our presentation.",
    "presenter.jpg",
    "talking_presenter.mp4"
)
```

## 后处理

### 放大结果

```python
import subprocess

def upscale_video(input_path, output_path):
    cmd = [
        "python", "-m", "realesrgan",
        "--input", input_path,
        "--output", output_path,
        "--scale", "2"
    ]
    subprocess.run(cmd, check=True)
```

### 重新添加音频

```bash

# 如果音频丢失，则将其加回
ffmpeg -i synced_video.mp4 -i original_audio.wav \
    -c:v copy -c:a aac \
    -map 0:v:0 -map 1:a:0 \
    final_output.mp4
```

## 故障排查

### 未检测到人脸

* 确保人脸清晰可见
* 光线良好
* 优先正面朝向
* 更高分辨率的输入

### 口型同步质量差

* 使用 wav2lip\_gan.pth
* 调整填充
* 检查音频采样率（建议 16kHz）

### 输出卡顿

* 增大 resize\_factor
* 禁用 nosmooth
* 使用更高质量的输入视频

## 性能

| 输入          | GPU      | 处理时间   |
| ----------- | -------- | ------ |
| 10 秒视频      | RTX 3060 | \~30 秒 |
| 10 秒视频      | RTX 4090 | \~15秒  |
| 30 秒视频      | RTX 4090 | \~45秒  |
| 图像 + 10 秒音频 | RTX 3090 | 约 20 秒 |

## 与 SadTalker 的比较

| 功能    | Wav2Lip | SadTalker |
| ----- | ------- | --------- |
| 口型准确度 | 优秀      | 好         |
| 头部运动  | 无       | 自然        |
| 表情    | 无       | 可控        |
| 速度    | 更快      | 较慢        |
| 最适合   | 配音      | 虚拟形象      |

## 成本估算

CLORE.AI 市场常见费率（截至 2024 年）：

| GPU       | 小时费率    | 日费率     | 4 小时会话  |
| --------- | ------- | ------- | ------- |
| RTX 3060  | \~$0.03 | \~$0.70 | \~$0.12 |
| RTX 3090  | \~$0.06 | \~$1.50 | \~$0.25 |
| RTX 4090  | \~$0.10 | \~$2.30 | \~$0.40 |
| A100 40GB | \~$0.17 | \~$4.00 | \~$0.70 |
| A100 80GB | \~$0.25 | \~$6.00 | \~$1.00 |

*价格因提供商和需求而异。请查看* [*CLORE.AI 市场*](https://clore.ai/marketplace) *以获取当前费率。*

**节省费用：**

* 使用 **竞价** 可中断工作市场——约三分之一的服务器将现货价格定得低于按需价格（中位数约优惠 13%），其余则与按需价格持平
* 使用 **CLORE** 代币支付
* 比较不同提供商的价格

## 下一步

* [SadTalker](/guides/guides_v2-zh/talking-heads/sadtalker.md) - 头部运动 + 嘴唇
* [XTTS](/guides/guides_v2-zh/yin-pin-yu-yu-yin/xtts-coqui.md) - 生成语音
* [RVC 变声克隆](/guides/guides_v2-zh/yin-pin-yu-yu-yin/rvc-voice-clone.md) - 语音转换


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.clore.ai/guides/guides_v2-zh/talking-heads/wav2lip.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.