# IP-Adapter

Use images as prompts for Stable Diffusion generation.

{% hint style="success" %}
All examples can be run on GPU servers rented through [CLORE.AI Marketplace](https://clore.ai/marketplace).
{% endhint %}

## Renting on CLORE.AI

1. Visit [CLORE.AI Marketplace](https://clore.ai/marketplace)
2. Filter by GPU type, VRAM, and price
3. Choose **On-Demand** (fixed rate) or **Spot** (bid price)
4. Configure your order:
   * Select Docker image
   * Set ports (TCP for SSH, HTTP for web UIs)
   * Add environment variables if needed
   * Enter startup command
5. Select payment: **CLORE**, **BTC**, or **USDT/USDC**
6. Create order and wait for deployment

### Access Your Server

* Find connection details in **My Orders**
* Web interfaces: Use the HTTP port URL
* SSH: `ssh -p <port> root@<proxy-address>`

## What is IP-Adapter?

IP-Adapter enables image prompting:

* Use reference images to guide generation
* Combine with text prompts
* Style transfer and content transfer
* Works with SD 1.5 and SDXL

## Adapter Types

| Adapter              | Use Case                | VRAM |
| -------------------- | ----------------------- | ---- |
| IP-Adapter           | General image prompting | 8GB  |
| IP-Adapter-Plus      | Higher quality          | 10GB |
| IP-Adapter-Face      | Face-focused            | 10GB |
| IP-Adapter-Full-Face | Full face details       | 12GB |

## Quick Deploy

**Docker Image:**

```
pytorch/pytorch:2.5.1-cuda12.4-cudnn9-devel
```

**Ports:**

```
22/tcp
7860/http
```

**Command:**

```bash
pip install diffusers transformers accelerate && \
python ip_adapter_app.py
```

## Accessing Your Service

After deployment, find your `http_pub` URL in **My Orders**:

1. Go to **My Orders** page
2. Click on your order
3. Find the `http_pub` URL (e.g., `abc123.clorecloud.net`)

Use `https://YOUR_HTTP_PUB_URL` instead of `localhost` in examples below.

## Installation

```bash
pip install diffusers transformers accelerate
```

## Basic Image Prompting

```python
from diffusers import AutoPipelineForText2Image
from diffusers.utils import load_image
import torch

pipe = AutoPipelineForText2Image.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    torch_dtype=torch.float16
).to("cuda")

# Load IP-Adapter
pipe.load_ip_adapter(
    "h94/IP-Adapter",
    subfolder="sdxl_models",
    weight_name="ip-adapter_sdxl.bin"
)

# Load reference image
ip_image = load_image("reference.jpg")

# Generate with image prompt
image = pipe(
    prompt="a cat in the same style",
    ip_adapter_image=ip_image,
    num_inference_steps=30
).images[0]

image.save("output.png")
```

## Style Transfer

```python
from diffusers import AutoPipelineForText2Image
from diffusers.utils import load_image
import torch

pipe = AutoPipelineForText2Image.from_pretrained(
    "runwayml/stable-diffusion-v1-5",
    torch_dtype=torch.float16
).to("cuda")

pipe.load_ip_adapter(
    "h94/IP-Adapter",
    subfolder="models",
    weight_name="ip-adapter_sd15.bin"
)

# Style reference (e.g., Van Gogh painting)
style_image = load_image("van_gogh_starry_night.jpg")

# Generate new content in that style
image = pipe(
    prompt="a modern city skyline",
    ip_adapter_image=style_image,
    num_inference_steps=30,
    guidance_scale=7.5
).images[0]

image.save("city_van_gogh.png")
```

## Face Adapter

For face-focused generation:

```python
from diffusers import AutoPipelineForText2Image
from diffusers.utils import load_image
import torch

pipe = AutoPipelineForText2Image.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    torch_dtype=torch.float16
).to("cuda")

# Load face-specific adapter
pipe.load_ip_adapter(
    "h94/IP-Adapter",
    subfolder="sdxl_models",
    weight_name="ip-adapter-plus-face_sdxl_vit-h.bin"
)

# Reference face
face_image = load_image("face_reference.jpg")

# Generate portrait
image = pipe(
    prompt="portrait painting, oil on canvas, museum quality",
    ip_adapter_image=face_image,
    num_inference_steps=30
).images[0]

image.save("portrait.png")
```

## Combining Multiple Images

```python
from diffusers import AutoPipelineForText2Image
from diffusers.utils import load_image
import torch

pipe = AutoPipelineForText2Image.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    torch_dtype=torch.float16
).to("cuda")

pipe.load_ip_adapter(
    "h94/IP-Adapter",
    subfolder="sdxl_models",
    weight_name="ip-adapter_sdxl.bin"
)

# Multiple reference images
images = [
    load_image("style1.jpg"),
    load_image("style2.jpg")
]

# Generate blending both
image = pipe(
    prompt="landscape painting",
    ip_adapter_image=images,
    num_inference_steps=30
).images[0]
```

## Scale Control

```python

# Set adapter strength
pipe.set_ip_adapter_scale(0.6)  # 0.0 to 1.0

# Low scale = more text prompt influence

# High scale = more image prompt influence

# Per-image scaling with multiple images
pipe.set_ip_adapter_scale([0.7, 0.3])
```

## With ControlNet

```python
from diffusers import (
    AutoPipelineForText2Image,
    ControlNetModel
)
from diffusers.utils import load_image
import torch

# Load ControlNet
controlnet = ControlNetModel.from_pretrained(
    "lllyasviel/control_v11p_sd15_canny",
    torch_dtype=torch.float16
)

pipe = AutoPipelineForText2Image.from_pretrained(
    "runwayml/stable-diffusion-v1-5",
    controlnet=controlnet,
    torch_dtype=torch.float16
).to("cuda")

# Load IP-Adapter
pipe.load_ip_adapter(
    "h94/IP-Adapter",
    subfolder="models",
    weight_name="ip-adapter_sd15.bin"
)

# Style image
style_image = load_image("style.jpg")

# Control image (edge map)
control_image = load_image("edges.png")

image = pipe(
    prompt="detailed illustration",
    image=control_image,
    ip_adapter_image=style_image,
    num_inference_steps=30
).images[0]
```

## Gradio Interface

```python
import gradio as gr
import torch
from diffusers import AutoPipelineForText2Image
from diffusers.utils import load_image

pipe = AutoPipelineForText2Image.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    torch_dtype=torch.float16
).to("cuda")

pipe.load_ip_adapter(
    "h94/IP-Adapter",
    subfolder="sdxl_models",
    weight_name="ip-adapter_sdxl.bin"
)

def generate(reference_image, prompt, negative_prompt, scale, steps):
    pipe.set_ip_adapter_scale(scale)

    image = pipe(
        prompt=prompt,
        negative_prompt=negative_prompt,
        ip_adapter_image=reference_image,
        num_inference_steps=steps
    ).images[0]

    return image

demo = gr.Interface(
    fn=generate,
    inputs=[
        gr.Image(type="pil", label="Reference Image"),
        gr.Textbox(label="Prompt", value="high quality"),
        gr.Textbox(label="Negative Prompt", value="ugly, blurry"),
        gr.Slider(0.0, 1.0, value=0.6, label="IP-Adapter Scale"),
        gr.Slider(10, 50, value=30, step=1, label="Steps")
    ],
    outputs=gr.Image(label="Generated Image"),
    title="IP-Adapter Image Prompting"
)

demo.launch(server_name="0.0.0.0", server_port=7860)
```

## Batch Style Transfer

```python
from diffusers import AutoPipelineForText2Image
from diffusers.utils import load_image
import torch
import os

pipe = AutoPipelineForText2Image.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    torch_dtype=torch.float16
).to("cuda")

pipe.load_ip_adapter(
    "h94/IP-Adapter",
    subfolder="sdxl_models",
    weight_name="ip-adapter_sdxl.bin"
)

# Style reference
style_image = load_image("art_style.jpg")

# Subjects to generate
subjects = [
    "a mountain landscape",
    "a city at night",
    "a forest in autumn",
    "an ocean sunset",
    "a snowy village"
]

output_dir = "./styled_outputs"
os.makedirs(output_dir, exist_ok=True)

for i, subject in enumerate(subjects):
    print(f"Generating {i+1}/{len(subjects)}: {subject}")

    image = pipe(
        prompt=subject,
        ip_adapter_image=style_image,
        num_inference_steps=30
    ).images[0]

    image.save(f"{output_dir}/styled_{i:03d}.png")
```

## Use Cases

### Product Photography Style

```python

# Reference: professional product photo
style = load_image("product_photo_reference.jpg")

image = pipe(
    prompt="red sneakers on white background",
    ip_adapter_image=style,
    num_inference_steps=30
).images[0]
```

### Artistic Style Transfer

```python

# Reference: famous painting
style = load_image("monet_painting.jpg")

image = pipe(
    prompt="portrait of a woman in a garden",
    ip_adapter_image=style,
    num_inference_steps=30
).images[0]
```

### Brand Consistency

```python

# Reference: brand style guide image
style = load_image("brand_style.jpg")

prompts = [
    "website hero banner",
    "social media post",
    "email newsletter header"
]

for prompt in prompts:
    image = pipe(prompt=prompt, ip_adapter_image=style).images[0]
```

## Memory Optimization

```python
pipe.enable_model_cpu_offload()
pipe.enable_vae_slicing()

# For very limited VRAM
pipe.enable_sequential_cpu_offload()
```

## Performance

| Model                  | GPU      | Time  |
| ---------------------- | -------- | ----- |
| SD 1.5 + IP-Adapter    | RTX 3090 | \~5s  |
| SDXL + IP-Adapter      | RTX 3090 | \~12s |
| SDXL + IP-Adapter      | RTX 4090 | \~8s  |
| SDXL + IP-Adapter-Plus | RTX 4090 | \~10s |

## Troubleshooting

### Style Not Applied

* Increase ip\_adapter\_scale
* Use clearer reference image
* Ensure adapter loaded correctly

### Too Much Reference Influence

* Decrease ip\_adapter\_scale
* More specific text prompt
* Increase guidance\_scale

### Memory Issues

* Enable CPU offload
* Use SD 1.5 instead of SDXL
* Lower resolution

## Cost Estimate

Typical CLORE.AI marketplace rates (as of 2024):

| GPU       | Hourly Rate | Daily Rate | 4-Hour Session |
| --------- | ----------- | ---------- | -------------- |
| RTX 3060  | \~$0.03     | \~$0.70    | \~$0.12        |
| RTX 3090  | \~$0.06     | \~$1.50    | \~$0.25        |
| RTX 4090  | \~$0.10     | \~$2.30    | \~$0.40        |
| A100 40GB | \~$0.17     | \~$4.00    | \~$0.70        |
| A100 80GB | \~$0.25     | \~$6.00    | \~$1.00        |

*Prices vary by provider and demand. Check* [*CLORE.AI Marketplace*](https://clore.ai/marketplace) *for current rates.*

**Save money:**

* Use **Spot** market for flexible workloads (often 30-50% cheaper)
* Pay with **CLORE** tokens
* Compare prices across different providers

## Next Steps

* [InstantID](https://docs.clore.ai/guides/face-and-identity/instantid) - Face identity
* [ControlNet](https://docs.clore.ai/guides/image-processing/controlnet-advanced) - Structure control
* Stable Diffusion WebUI - IP-Adapter extension