# IP-Adapter

Use images as prompts for Stable Diffusion generation.

{% hint style="success" %}
All examples can be run on GPU servers rented through [CLORE.AI Marketplace](https://clore.ai/marketplace).
{% endhint %}

## Renting on CLORE.AI

1. Visit [CLORE.AI Marketplace](https://clore.ai/marketplace)
2. Filter by GPU type, VRAM, and price
3. Choose **On-Demand** (fixed rate) or **Spot** (bid price)
4. Configure your order:
   * Select Docker image
   * Set ports (TCP for SSH, HTTP for web UIs)
   * Add environment variables if needed
   * Enter startup command
5. Select payment: **CLORE**, **BTC**, or **USDT/USDC**
6. Create order and wait for deployment

### Access Your Server

* Find connection details in **My Orders**
* Web interfaces: Use the HTTP port URL
* SSH: `ssh -p <port> root@<proxy-address>`

## What is IP-Adapter?

IP-Adapter enables image prompting:

* Use reference images to guide generation
* Combine with text prompts
* Style transfer and content transfer
* Works with SD 1.5 and SDXL

## Adapter Types

| Adapter              | Use Case                | VRAM |
| -------------------- | ----------------------- | ---- |
| IP-Adapter           | General image prompting | 8GB  |
| IP-Adapter-Plus      | Higher quality          | 10GB |
| IP-Adapter-Face      | Face-focused            | 10GB |
| IP-Adapter-Full-Face | Full face details       | 12GB |

## Quick Deploy

**Docker Image:**

```
pytorch/pytorch:2.5.1-cuda12.4-cudnn9-devel
```

**Ports:**

```
22/tcp
7860/http
```

**Command:**

```bash
pip install diffusers transformers accelerate && \
python ip_adapter_app.py
```

## Accessing Your Service

After deployment, find your `http_pub` URL in **My Orders**:

1. Go to **My Orders** page
2. Click on your order
3. Find the `http_pub` URL (e.g., `abc123.clorecloud.net`)

Use `https://YOUR_HTTP_PUB_URL` instead of `localhost` in examples below.

## Installation

```bash
pip install diffusers transformers accelerate
```

## Basic Image Prompting

```python
from diffusers import AutoPipelineForText2Image
from diffusers.utils import load_image
import torch

pipe = AutoPipelineForText2Image.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    torch_dtype=torch.float16
).to("cuda")

# Load IP-Adapter
pipe.load_ip_adapter(
    "h94/IP-Adapter",
    subfolder="sdxl_models",
    weight_name="ip-adapter_sdxl.bin"
)

# Load reference image
ip_image = load_image("reference.jpg")

# Generate with image prompt
image = pipe(
    prompt="a cat in the same style",
    ip_adapter_image=ip_image,
    num_inference_steps=30
).images[0]

image.save("output.png")
```

## Style Transfer

```python
from diffusers import AutoPipelineForText2Image
from diffusers.utils import load_image
import torch

pipe = AutoPipelineForText2Image.from_pretrained(
    "runwayml/stable-diffusion-v1-5",
    torch_dtype=torch.float16
).to("cuda")

pipe.load_ip_adapter(
    "h94/IP-Adapter",
    subfolder="models",
    weight_name="ip-adapter_sd15.bin"
)

# Style reference (e.g., Van Gogh painting)
style_image = load_image("van_gogh_starry_night.jpg")

# Generate new content in that style
image = pipe(
    prompt="a modern city skyline",
    ip_adapter_image=style_image,
    num_inference_steps=30,
    guidance_scale=7.5
).images[0]

image.save("city_van_gogh.png")
```

## Face Adapter

For face-focused generation:

```python
from diffusers import AutoPipelineForText2Image
from diffusers.utils import load_image
import torch

pipe = AutoPipelineForText2Image.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    torch_dtype=torch.float16
).to("cuda")

# Load face-specific adapter
pipe.load_ip_adapter(
    "h94/IP-Adapter",
    subfolder="sdxl_models",
    weight_name="ip-adapter-plus-face_sdxl_vit-h.bin"
)

# Reference face
face_image = load_image("face_reference.jpg")

# Generate portrait
image = pipe(
    prompt="portrait painting, oil on canvas, museum quality",
    ip_adapter_image=face_image,
    num_inference_steps=30
).images[0]

image.save("portrait.png")
```

## Combining Multiple Images

```python
from diffusers import AutoPipelineForText2Image
from diffusers.utils import load_image
import torch

pipe = AutoPipelineForText2Image.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    torch_dtype=torch.float16
).to("cuda")

pipe.load_ip_adapter(
    "h94/IP-Adapter",
    subfolder="sdxl_models",
    weight_name="ip-adapter_sdxl.bin"
)

# Multiple reference images
images = [
    load_image("style1.jpg"),
    load_image("style2.jpg")
]

# Generate blending both
image = pipe(
    prompt="landscape painting",
    ip_adapter_image=images,
    num_inference_steps=30
).images[0]
```

## Scale Control

```python

# Set adapter strength
pipe.set_ip_adapter_scale(0.6)  # 0.0 to 1.0

# Low scale = more text prompt influence

# High scale = more image prompt influence

# Per-image scaling with multiple images
pipe.set_ip_adapter_scale([0.7, 0.3])
```

## With ControlNet

```python
from diffusers import (
    AutoPipelineForText2Image,
    ControlNetModel
)
from diffusers.utils import load_image
import torch

# Load ControlNet
controlnet = ControlNetModel.from_pretrained(
    "lllyasviel/control_v11p_sd15_canny",
    torch_dtype=torch.float16
)

pipe = AutoPipelineForText2Image.from_pretrained(
    "runwayml/stable-diffusion-v1-5",
    controlnet=controlnet,
    torch_dtype=torch.float16
).to("cuda")

# Load IP-Adapter
pipe.load_ip_adapter(
    "h94/IP-Adapter",
    subfolder="models",
    weight_name="ip-adapter_sd15.bin"
)

# Style image
style_image = load_image("style.jpg")

# Control image (edge map)
control_image = load_image("edges.png")

image = pipe(
    prompt="detailed illustration",
    image=control_image,
    ip_adapter_image=style_image,
    num_inference_steps=30
).images[0]
```

## Gradio Interface

```python
import gradio as gr
import torch
from diffusers import AutoPipelineForText2Image
from diffusers.utils import load_image

pipe = AutoPipelineForText2Image.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    torch_dtype=torch.float16
).to("cuda")

pipe.load_ip_adapter(
    "h94/IP-Adapter",
    subfolder="sdxl_models",
    weight_name="ip-adapter_sdxl.bin"
)

def generate(reference_image, prompt, negative_prompt, scale, steps):
    pipe.set_ip_adapter_scale(scale)

    image = pipe(
        prompt=prompt,
        negative_prompt=negative_prompt,
        ip_adapter_image=reference_image,
        num_inference_steps=steps
    ).images[0]

    return image

demo = gr.Interface(
    fn=generate,
    inputs=[
        gr.Image(type="pil", label="Reference Image"),
        gr.Textbox(label="Prompt", value="high quality"),
        gr.Textbox(label="Negative Prompt", value="ugly, blurry"),
        gr.Slider(0.0, 1.0, value=0.6, label="IP-Adapter Scale"),
        gr.Slider(10, 50, value=30, step=1, label="Steps")
    ],
    outputs=gr.Image(label="Generated Image"),
    title="IP-Adapter Image Prompting"
)

demo.launch(server_name="0.0.0.0", server_port=7860)
```

## Batch Style Transfer

```python
from diffusers import AutoPipelineForText2Image
from diffusers.utils import load_image
import torch
import os

pipe = AutoPipelineForText2Image.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    torch_dtype=torch.float16
).to("cuda")

pipe.load_ip_adapter(
    "h94/IP-Adapter",
    subfolder="sdxl_models",
    weight_name="ip-adapter_sdxl.bin"
)

# Style reference
style_image = load_image("art_style.jpg")

# Subjects to generate
subjects = [
    "a mountain landscape",
    "a city at night",
    "a forest in autumn",
    "an ocean sunset",
    "a snowy village"
]

output_dir = "./styled_outputs"
os.makedirs(output_dir, exist_ok=True)

for i, subject in enumerate(subjects):
    print(f"Generating {i+1}/{len(subjects)}: {subject}")

    image = pipe(
        prompt=subject,
        ip_adapter_image=style_image,
        num_inference_steps=30
    ).images[0]

    image.save(f"{output_dir}/styled_{i:03d}.png")
```

## Use Cases

### Product Photography Style

```python

# Reference: professional product photo
style = load_image("product_photo_reference.jpg")

image = pipe(
    prompt="red sneakers on white background",
    ip_adapter_image=style,
    num_inference_steps=30
).images[0]
```

### Artistic Style Transfer

```python

# Reference: famous painting
style = load_image("monet_painting.jpg")

image = pipe(
    prompt="portrait of a woman in a garden",
    ip_adapter_image=style,
    num_inference_steps=30
).images[0]
```

### Brand Consistency

```python

# Reference: brand style guide image
style = load_image("brand_style.jpg")

prompts = [
    "website hero banner",
    "social media post",
    "email newsletter header"
]

for prompt in prompts:
    image = pipe(prompt=prompt, ip_adapter_image=style).images[0]
```

## Memory Optimization

```python
pipe.enable_model_cpu_offload()
pipe.enable_vae_slicing()

# For very limited VRAM
pipe.enable_sequential_cpu_offload()
```

## Performance

| Model                  | GPU      | Time  |
| ---------------------- | -------- | ----- |
| SD 1.5 + IP-Adapter    | RTX 3090 | \~5s  |
| SDXL + IP-Adapter      | RTX 3090 | \~12s |
| SDXL + IP-Adapter      | RTX 4090 | \~8s  |
| SDXL + IP-Adapter-Plus | RTX 4090 | \~10s |

## Troubleshooting

### Style Not Applied

* Increase ip\_adapter\_scale
* Use clearer reference image
* Ensure adapter loaded correctly

### Too Much Reference Influence

* Decrease ip\_adapter\_scale
* More specific text prompt
* Increase guidance\_scale

### Memory Issues

* Enable CPU offload
* Use SD 1.5 instead of SDXL
* Lower resolution

## Cost Estimate

Typical CLORE.AI marketplace rates (as of 2024):

| GPU       | Hourly Rate | Daily Rate | 4-Hour Session |
| --------- | ----------- | ---------- | -------------- |
| RTX 3060  | \~$0.03     | \~$0.70    | \~$0.12        |
| RTX 3090  | \~$0.06     | \~$1.50    | \~$0.25        |
| RTX 4090  | \~$0.10     | \~$2.30    | \~$0.40        |
| A100 40GB | \~$0.17     | \~$4.00    | \~$0.70        |
| A100 80GB | \~$0.25     | \~$6.00    | \~$1.00        |

*Prices vary by provider and demand. Check* [*CLORE.AI Marketplace*](https://clore.ai/marketplace) *for current rates.*

**Save money:**

* Use **Spot** market for flexible workloads (often 30-50% cheaper)
* Pay with **CLORE** tokens
* Compare prices across different providers

## Next Steps

* [InstantID](/guides/face-and-identity/instantid.md) - Face identity
* [ControlNet](/guides/image-processing/controlnet-advanced.md) - Structure control
* Stable Diffusion WebUI - IP-Adapter extension


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.clore.ai/guides/face-and-identity/ip-adapter.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.