> For the complete documentation index, see [llms.txt](https://docs.clore.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.clore.ai/guides/guides_v2-zh/ren-lian-yu-shen-fen/ip-adapter.md).

# IP-Adapter

使用 IP-Adapter 将图像作为 Stable Diffusion 的提示词

使用图像作为 Stable Diffusion 生成的提示。

{% hint style="success" %}
所有示例都可以在通过以下方式租用的 GPU 服务器上运行 [CLORE.AI 市场](https://clore.ai/marketplace).
{% endhint %}

## 在 CLORE.AI 上租用

1. 访问 [CLORE.AI 市场](https://clore.ai/marketplace)
2. 按 GPU 类型、VRAM 和价格筛选
3. 选择 **按需** （固定费率）或 **竞价** （出价）
4. 配置你的订单：
   * 选择 Docker 镜像
   * 设置端口（SSH 用 TCP，Web UI 用 HTTP）
   * 如有需要，添加环境变量
   * 输入启动命令
5. 选择支付方式： **CLORE**, **BTC**，或 **USDT/USDC**
6. 创建订单并等待部署

### 访问你的服务器

* 在以下位置查找连接信息 **我的订单**
* Web 界面：使用 HTTP 端口 URL
* SSH： `ssh -p <port> root@<proxy-address>`

## 什么是 IP-Adapter？

IP-Adapter 支持图像提示：

* 使用参考图像引导生成
* 与文本提示结合
* 风格迁移和内容迁移
* 适用于 SD 1.5 和 SDXL

## 适配器类型

| 适配器                  | 使用场景   | 显存   |
| -------------------- | ------ | ---- |
| IP-Adapter           | 通用图像提示 | 8GB  |
| IP-Adapter-Plus      | 更高质量   | 10GB |
| IP-Adapter-Face      | 专注人脸   | 10GB |
| IP-Adapter-Full-Face | 完整人脸细节 | 12GB |

## 快速部署

**Docker 镜像：**

```
pytorch/pytorch:2.11.0-cuda12.8-cudnn9-devel
```

**端口：**

```
22/tcp
7860/http
```

**命令：**

```bash
pip install diffusers transformers accelerate && \\
python ip_adapter_app.py
```

## 访问你的服务

部署后，找到你的 `http_pub` URL 在 **我的订单**:

1. 前往 **我的订单** 页面
2. 点击你的订单
3. 找到 `http_pub` URL（例如， `abc123.clorecloud.net`)

使用 `https://YOUR_HTTP_PUB_URL` 替代 `localhost` 在下面的示例中。

## 安装

```bash
pip install diffusers transformers accelerate
```

## 基础图像提示

```python
from diffusers import AutoPipelineForText2Image
from diffusers.utils import load_image
import torch

pipe = AutoPipelineForText2Image.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    torch_dtype=torch.float16
).to("cuda")

# 加载 IP-Adapter
pipe.load_ip_adapter(
    "h94/IP-Adapter",
    subfolder="sdxl_models",
    weight_name="ip-adapter_sdxl.bin"
)

# 加载参考图像
ip_image = load_image("reference.jpg")

# 使用图像提示生成
image = pipe(
    prompt="一只相同风格的猫",
    ip_adapter_image=ip_image,
    num_inference_steps=30
).images[0]

image.save("output.png")
```

## 风格迁移

```python
from diffusers import AutoPipelineForText2Image
from diffusers.utils import load_image
import torch

pipe = AutoPipelineForText2Image.from_pretrained(
    "runwayml/stable-diffusion-v1-5",
    torch_dtype=torch.float16
).to("cuda")

pipe.load_ip_adapter(
    "h94/IP-Adapter",
    subfolder="models",
    weight_name="ip-adapter_sd15.bin"
)

# 风格参考（例如，梵高的画）
style_image = load_image("van_gogh_starry_night.jpg")

# 以该风格生成新内容
image = pipe(
    prompt="现代城市天际线",
    ip_adapter_image=style_image,
    num_inference_steps=30,
    guidance_scale=7.5
).images[0]

image.save("city_van_gogh.png")
```

## 人脸适配器

用于专注人脸的生成：

```python
from diffusers import AutoPipelineForText2Image
from diffusers.utils import load_image
import torch

pipe = AutoPipelineForText2Image.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    torch_dtype=torch.float16
).to("cuda")

# 加载人脸专用适配器
pipe.load_ip_adapter(
    "h94/IP-Adapter",
    subfolder="sdxl_models",
    weight_name="ip-adapter-plus-face_sdxl_vit-h.bin"
)

# 参考人脸
face_image = load_image("face_reference.jpg")

# 生成肖像
image = pipe(
    prompt="肖像画，油画布面，博物馆级品质",
    ip_adapter_image=face_image,
    num_inference_steps=30
).images[0]

image.save("portrait.png")
```

## 结合多张图像

```python
from diffusers import AutoPipelineForText2Image
from diffusers.utils import load_image
import torch

pipe = AutoPipelineForText2Image.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    torch_dtype=torch.float16
).to("cuda")

pipe.load_ip_adapter(
    "h94/IP-Adapter",
    subfolder="sdxl_models",
    weight_name="ip-adapter_sdxl.bin"
)

# 多张参考图像
images = [
    load_image("style1.jpg"),
    load_image("style2.jpg")
]

# 生成融合两者的结果
image = pipe(
    prompt="风景画",
    ip_adapter_image=images,
    num_inference_steps=30
).images[0]
```

## 缩放控制

```python

# 设置适配器强度
pipe.set_ip_adapter_scale(0.6)  # 0.0 到 1.0

# 低缩放 = 更多文本提示影响

# 高缩放 = 更多图像提示影响

# 使用多张图像时的按图缩放
pipe.set_ip_adapter_scale([0.7, 0.3])
```

## 结合 ControlNet

```python
from diffusers import (
    AutoPipelineForText2Image,
    ControlNetModel
)
from diffusers.utils import load_image
import torch

# 加载 ControlNet
controlnet = ControlNetModel.from_pretrained(
    "lllyasviel/control_v11p_sd15_canny",
    torch_dtype=torch.float16
)

pipe = AutoPipelineForText2Image.from_pretrained(
    "runwayml/stable-diffusion-v1-5",
    controlnet=controlnet,
    torch_dtype=torch.float16
).to("cuda")

# 加载 IP-Adapter
pipe.load_ip_adapter(
    "h94/IP-Adapter",
    subfolder="models",
    weight_name="ip-adapter_sd15.bin"
)

# 风格图像
style_image = load_image("style.jpg")

# 控制图像（边缘图）
control_image = load_image("edges.png")

image = pipe(
    prompt="细致插画",
    image=control_image,
    ip_adapter_image=style_image,
    num_inference_steps=30
).images[0]
```

## Gradio 界面

```python
import gradio as gr
import torch
from diffusers import AutoPipelineForText2Image
from diffusers.utils import load_image

pipe = AutoPipelineForText2Image.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    torch_dtype=torch.float16
).to("cuda")

pipe.load_ip_adapter(
    "h94/IP-Adapter",
    subfolder="sdxl_models",
    weight_name="ip-adapter_sdxl.bin"
)

def generate(reference_image, prompt, negative_prompt, scale, steps):
    pipe.set_ip_adapter_scale(scale)

    image = pipe(
        prompt=prompt,
        negative_prompt=negative_prompt,
        ip_adapter_image=reference_image,
        num_inference_steps=steps
    ).images[0]

    return image

demo = gr.Interface(
    fn=generate,
    inputs=[
        gr.Image(type="pil", label="参考图像"),
        gr.Textbox(label="提示词", value="高质量"),
        gr.Textbox(label="负面提示词", value="丑陋，模糊"),
        gr.Slider(0.0, 1.0, value=0.6, label="IP-Adapter 缩放"),
        gr.Slider(10, 50, value=30, step=1, label="步数")
    ],
    outputs=gr.Image(label="生成的图像"),
    title="IP-Adapter 图像提示"
)

demo.launch(server_name="0.0.0.0", server_port=7860)
```

## 批量风格迁移

```python
from diffusers import AutoPipelineForText2Image
from diffusers.utils import load_image
import torch
import os

pipe = AutoPipelineForText2Image.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    torch_dtype=torch.float16
).to("cuda")

pipe.load_ip_adapter(
    "h94/IP-Adapter",
    subfolder="sdxl_models",
    weight_name="ip-adapter_sdxl.bin"
)

# 风格参考
style_image = load_image("art_style.jpg")

# 要生成的主题
subjects = [
    "一片山地风景",
    "夜晚的城市",
    "秋天的森林",
    "海洋日落",
    "一个雪中的村庄"
]

output_dir = "./styled_outputs"
os.makedirs(output_dir, exist_ok=True)

for i, subject in enumerate(subjects):
    print(f"正在生成 {i+1}/{len(subjects)}：{subject}")

    image = pipe(
        prompt=subject,
        ip_adapter_image=style_image,
        num_inference_steps=30
    ).images[0]

    image.save(f"{output_dir}/styled_{i:03d}.png")
```

## 应用场景

### 产品摄影风格

```python

# 参考：专业产品照片
style = load_image("product_photo_reference.jpg")

image = pipe(
    prompt="白色背景上的红色运动鞋",
    ip_adapter_image=style,
    num_inference_steps=30
).images[0]
```

### 艺术风格迁移

```python

# 参考：著名画作
style = load_image("monet_painting.jpg")

image = pipe(
    prompt="花园中的一位女性肖像",
    ip_adapter_image=style,
    num_inference_steps=30
).images[0]
```

### 品牌一致性

```python

# 参考：品牌风格指南图像
style = load_image("brand_style.jpg")

prompts = [
    "网站首屏横幅",
    "社交媒体帖子",
    "电子邮件新闻简报页眉"
]

for prompt in prompts:
    image = pipe(prompt=prompt, ip_adapter_image=style).images[0]
```

## 内存优化

```python
pipe.enable_model_cpu_offload()
pipe.enable_vae_slicing()

# 适用于显存极少的情况
pipe.enable_sequential_cpu_offload()
```

## 性能

| 模型                     | GPU      | 时间     |
| ---------------------- | -------- | ------ |
| SD 1.5 + IP-Adapter    | RTX 3090 | 约 5 秒  |
| SDXL + IP-Adapter      | RTX 3090 | \~12 秒 |
| SDXL + IP-Adapter      | RTX 4090 | \~8s   |
| SDXL + IP-Adapter-Plus | RTX 4090 | \~10 秒 |

## 故障排查

### 未应用风格

* 提高 ip\_adapter\_scale
* 使用更清晰的参考图像
* 确保适配器已正确加载

### 参考图像影响过强

* 降低 ip\_adapter\_scale
* 使用更具体的文本提示
* 提高 guidance\_scale

### 内存问题

* 启用 CPU 卸载
* 使用 SD 1.5 代替 SDXL
* 降低分辨率

## 成本估算

CLORE.AI 市场常见费率（截至 2024 年）：

| GPU       | 小时费率    | 日费率     | 4 小时会话  |
| --------- | ------- | ------- | ------- |
| RTX 3060  | \~$0.03 | \~$0.70 | \~$0.12 |
| RTX 3090  | \~$0.06 | \~$1.50 | \~$0.25 |
| RTX 4090  | \~$0.10 | \~$2.30 | \~$0.40 |
| A100 40GB | \~$0.17 | \~$4.00 | \~$0.70 |
| A100 80GB | \~$0.25 | \~$6.00 | \~$1.00 |

*价格因提供商和需求而异。请查看* [*CLORE.AI 市场*](https://clore.ai/marketplace) *以获取当前费率。*

**节省费用：**

* 使用 **竞价** 可中断工作市场——约三分之一的服务器将现货价格定得低于按需价格（中位数约优惠 13%），其余则与按需价格持平
* 使用 **CLORE** 代币支付
* 比较不同提供商的价格

## 下一步

* [InstantID](/guides/guides_v2-zh/ren-lian-yu-shen-fen/instantid.md) - 人脸身份
* [ControlNet](/guides/guides_v2-zh/tu-xiang-chu-li/controlnet-advanced.md) - 结构控制
* Stable Diffusion WebUI - IP-Adapter 扩展


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.clore.ai/guides/guides_v2-zh/ren-lian-yu-shen-fen/ip-adapter.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.