> For the complete documentation index, see [llms.txt](https://docs.clore.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.clore.ai/guides/guides_v2-zh/tu-xiang-chu-li/segment-anything.md).

# Segment Anything

在 Clore.ai GPU 上使用 Meta 的 SAM 进行精确图像分割

使用 Meta 的 SAM 在 GPU 上进行精确图像分割。

{% hint style="success" %}
所有示例都可以在通过以下方式租用的 GPU 服务器上运行 [CLORE.AI 市场](https://clore.ai/marketplace).
{% endhint %}

## 在 CLORE.AI 上租用

1. 访问 [CLORE.AI 市场](https://clore.ai/marketplace)
2. 按 GPU 类型、VRAM 和价格筛选
3. 选择 **按需** （固定费率）或 **竞价** （出价）
4. 配置你的订单：
   * 选择 Docker 镜像
   * 设置端口（SSH 用 TCP，Web UI 用 HTTP）
   * 如有需要，添加环境变量
   * 输入启动命令
5. 选择支付方式： **CLORE**, **BTC**，或 **USDT/USDC**
6. 创建订单并等待部署

### 访问你的服务器

* 在以下位置查找连接信息 **我的订单**
* Web 界面：使用 HTTP 端口 URL
* SSH： `ssh -p <port> root@<proxy-address>`

## 什么是 SAM？

Segment Anything Model（SAM）可以：

* 分割图像中的任意对象
* 配合提示（点、框、文本）工作
* 生成自动掩码
* 处理任何图像类型

## 模型变体

| 模型        | 显存   | 质量 | 速度 |
| --------- | ---- | -- | -- |
| SAM-H（超大） | 8GB  | 最佳 | 慢  |
| SAM-L（大）  | 6GB  | 很高 | 中等 |
| SAM-B（基础） | 4GB  | 好  | 快  |
| SAM2      | 8GB+ | 最佳 | 中等 |

## 快速部署

**Docker 镜像：**

```
pytorch/pytorch:2.11.0-cuda12.8-cudnn9-devel
```

**端口：**

```
22/tcp
7860/http
```

**命令：**

```bash
pip install segment-anything gradio opencv-python && \
wget https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth && \
python -c "
import gradio as gr
import numpy as np
from segment_anything import sam_model_registry, SamPredictor
import cv2

sam = sam_model_registry['vit_h'](checkpoint='sam_vit_h_4b8939.pth').cuda()
predictor = SamPredictor(sam)

def segment(image, evt: gr.SelectData):
    predictor.set_image(image)
    point = np.array([[evt.index[0], evt.index[1]]])
    masks, _, _ = predictor.predict(point_coords=point, point_labels=np.array([1]))
    mask = masks[0]
    colored = np.zeros_like(image)
    colored[mask] = [255, 0, 0]
    result = cv2.addWeighted(image, 0.7, colored, 0.3, 0)
    return result

demo = gr.Interface(fn=segment, inputs=gr.Image(), outputs=gr.Image(), title='点击进行分割')
demo.launch(server_name='0.0.0.0', server_port=7860)
"
```

## 访问你的服务

部署后，找到你的 `http_pub` URL 在 **我的订单**:

1. 前往 **我的订单** 页面
2. 点击你的订单
3. 找到 `http_pub` URL（例如， `abc123.clorecloud.net`)

使用 `https://YOUR_HTTP_PUB_URL` 替代 `localhost` 在下面的示例中。

## 安装

```bash
pip install segment-anything opencv-python
```

### 下载模型

```bash

# SAM-H（最佳质量）
wget https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth

# SAM-L（均衡）
wget https://dl.fbaipublicfiles.com/segment_anything/sam_vit_l_0b3195.pth

# SAM-B（快速）
wget https://dl.fbaipublicfiles.com/segment_anything/sam_vit_b_01ec64.pth
```

## Python API

### 使用点进行基础分割

```python
from segment_anything import sam_model_registry, SamPredictor
import cv2
import numpy as np

# 加载模型
sam = sam_model_registry["vit_h"](checkpoint="sam_vit_h_4b8939.pth")
sam.to("cuda")

predictor = SamPredictor(sam)

# 加载图像
image = cv2.imread("photo.jpg")
image_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

# 设置图像
predictor.set_image(image_rgb)

# 使用点提示进行分割
input_point = np.array([[500, 375]])  # x, y 坐标
input_label = np.array([1])  # 1 = 前景，0 = 背景

masks, scores, logits = predictor.predict(
    point_coords=input_point,
    point_labels=input_label,
    multimask_output=True
)

# 获取最佳遮罩
best_mask = masks[np.argmax(scores)]

# 保存掩码
cv2.imwrite("mask.png", best_mask.astype(np.uint8) * 255)
```

### 框提示

```python

# 使用边界框进行分割
input_box = np.array([100, 100, 400, 400])  # x1, y1, x2, y2

masks, scores, _ = predictor.predict(
    box=input_box,
    multimask_output=False
)
```

### 多个点

```python

# 多个前景/背景点
input_points = np.array([
    [500, 375],   # 点 1
    [550, 400],   # 点 2
    [100, 100],   # 背景点
])
input_labels = np.array([1, 1, 0])  # 1=前景，0=背景

masks, scores, _ = predictor.predict(
    point_coords=input_points,
    point_labels=input_labels,
    multimask_output=True
)
```

### 框 + 点组合

```python
masks, scores, _ = predictor.predict(
    point_coords=input_point,
    point_labels=input_label,
    box=input_box,
    multimask_output=False
)
```

## 自动掩码生成

生成所有可能的掩码：

```python
from segment_anything import SamAutomaticMaskGenerator
import cv2

sam = sam_model_registry["vit_h"](checkpoint="sam_vit_h_4b8939.pth")
sam.to("cuda")

mask_generator = SamAutomaticMaskGenerator(
    model=sam,
    points_per_side=32,
    pred_iou_thresh=0.86,
    stability_score_thresh=0.92,
    crop_n_layers=1,
    crop_n_points_downscale_factor=2,
    min_mask_region_area=100
)

image = cv2.imread("photo.jpg")
image_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

masks = mask_generator.generate(image_rgb)

# 每个掩码包含：

# - 'segmentation'：二值掩码

# - 'area'：掩码面积（像素）

# - 'bbox'：边界框

# - 'predicted_iou'：质量分数

# - 'stability_score'：稳定性分数

print(f"找到 {len(masks)} 个掩码")
```

### 可视化所有掩码

```python
import matplotlib.pyplot as plt

def show_masks(image, masks):
    plt.figure(figsize=(20, 20))
    plt.imshow(image)

    sorted_masks = sorted(masks, key=lambda x: x['area'], reverse=True)

    for mask in sorted_masks:
        m = mask['segmentation']
        color = np.random.random(3)
        colored = np.zeros((*m.shape, 4))
        colored[m] = [*color, 0.5]
        plt.imshow(colored)

    plt.axis('off')
    plt.savefig('all_masks.png')

show_masks(image_rgb, masks)
```

## SAM 2（最新版本）

```bash
pip install sam2
```

```python
from sam2.sam2_image_predictor import SAM2ImagePredictor

predictor = SAM2ImagePredictor.from_pretrained("facebook/sam2-hiera-large")

with torch.inference_mode():
    predictor.set_image(image)
    masks, scores, _ = predictor.predict(
        point_coords=points,
        point_labels=labels
    )
```

## 去除背景

```python
from segment_anything import sam_model_registry, SamPredictor
import cv2
import numpy as np

sam = sam_model_registry["vit_h"](checkpoint="sam_vit_h_4b8939.pth")
sam.to("cuda")
predictor = SamPredictor(sam)

def remove_background(image_path, point):
    image = cv2.imread(image_path)
    image_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

    predictor.set_image(image_rgb)

    masks, scores, _ = predictor.predict(
        point_coords=np.array([point]),
        point_labels=np.array([1]),
        multimask_output=True
    )

    best_mask = masks[np.argmax(scores)]

    # 创建 RGBA 图像
    result = cv2.cvtColor(image, cv2.COLOR_BGR2BGRA)
    result[:, :, 3] = best_mask.astype(np.uint8) * 255

    return result

# 点击要保留的对象
result = remove_background("photo.jpg", [400, 300])
cv2.imwrite("no_background.png", result)
```

## 提取对象

```python
def extract_object(image_path, point):
    image = cv2.imread(image_path)
    image_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

    predictor.set_image(image_rgb)

    masks, scores, _ = predictor.predict(
        point_coords=np.array([point]),
        point_labels=np.array([1]),
        multimask_output=True
    )

    best_mask = masks[np.argmax(scores)]

    # 获取边界框
    rows = np.any(best_mask, axis=1)
    cols = np.any(best_mask, axis=0)
    y1, y2 = np.where(rows)[0][[0, -1]]
    x1, x2 = np.where(cols)[0][[0, -1]]

    # 裁剪
    cropped = image[y1:y2+1, x1:x2+1]
    mask_cropped = best_mask[y1:y2+1, x1:x2+1]

    # 应用掩码
    result = cv2.cvtColor(cropped, cv2.COLOR_BGR2BGRA)
    result[:, :, 3] = mask_cropped.astype(np.uint8) * 255

    return result
```

## 批量处理

```python
import os
from segment_anything import sam_model_registry, SamAutomaticMaskGenerator
import cv2
import json

sam = sam_model_registry["vit_h"](checkpoint="sam_vit_h_4b8939.pth")
sam.to("cuda")

mask_generator = SamAutomaticMaskGenerator(sam)

input_dir = "./images"
output_dir = "./segmented"
os.makedirs(output_dir, exist_ok=True)

for filename in os.listdir(input_dir):
    if filename.lower().endswith(('.png', '.jpg', '.jpeg')):
        image = cv2.imread(os.path.join(input_dir, filename))
        image_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

        masks = mask_generator.generate(image_rgb)

        # 将掩码保存为 JSON
        mask_data = []
        for i, mask in enumerate(masks):
            mask_data.append({
                'id': i,
                'area': int(mask['area']),
                'bbox': mask['bbox'],
                'score': float(mask['predicted_iou'])
            })

            # 保存单个掩码
            cv2.imwrite(
                os.path.join(output_dir, f"{filename}_mask_{i}.png"),
                mask['segmentation'].astype(np.uint8) * 255
            )

        with open(os.path.join(output_dir, f"{filename}_masks.json"), 'w') as f:
            json.dump(mask_data, f)
```

## API 服务器

```python
from fastapi import FastAPI, UploadFile
from fastapi.responses import Response
from segment_anything import sam_model_registry, SamPredictor
import cv2
import numpy as np
import json

app = FastAPI()

sam = sam_model_registry["vit_h"](checkpoint="sam_vit_h_4b8939.pth")
sam.to("cuda")
predictor = SamPredictor(sam)

@app.post("/segment")
async def segment(file: UploadFile, x: int, y: int):
    contents = await file.read()
    nparr = np.frombuffer(contents, np.uint8)
    image = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
    image_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

    predictor.set_image(image_rgb)

    masks, scores, _ = predictor.predict(
        point_coords=np.array([[x, y]]),
        point_labels=np.array([1]),
        multimask_output=True
    )

    best_mask = masks[np.argmax(scores)]

    _, encoded = cv2.imencode('.png', best_mask.astype(np.uint8) * 255)
    return Response(content=encoded.tobytes(), media_type="image/png")
```

## 与 Stable Diffusion 集成

使用 SAM 掩码进行修复：

```python

# 使用 SAM 生成掩码
predictor.set_image(image)
masks, scores, _ = predictor.predict(point_coords=point, point_labels=label)
mask = masks[np.argmax(scores)]

# 用于 SD 修复
from diffusers import StableDiffusionInpaintPipeline

pipe = StableDiffusionInpaintPipeline.from_pretrained("runwayml/stable-diffusion-inpainting")
pipe.to("cuda")

result = pipe(
    prompt="一辆红色跑车",
    image=image,
    mask_image=mask
).images[0]
```

## 性能

| 模型    | 图像尺寸      | GPU      | 时间      |
| ----- | --------- | -------- | ------- |
| SAM-H | 1024x1024 | RTX 3090 | \~0.5 秒 |
| SAM-L | 1024x1024 | RTX 3090 | \~0.3 秒 |
| SAM-B | 1024x1024 | RTX 3090 | \~0.2秒  |
| SAM2  | 1024x1024 | RTX 4090 | \~0.3 秒 |

## 内存优化

```python

# 对于有限的显存
sam = sam_model_registry["vit_b"](checkpoint="sam_vit_b_01ec64.pth")  # 使用更小的模型

# 或减少自动生成点数
mask_generator = SamAutomaticMaskGenerator(
    model=sam,
    points_per_side=16,  # 从 32 减少
)
```

## 故障排查

### CUDA 显存不足

* 使用 SAM-B 而不是 SAM-H
* 处理前减小图像尺寸
* 清除缓存： `torch.cuda.empty_cache()`

### 分割效果差

* 添加更多点（前景 + 背景）
* 使用框提示以获得更好的引导
* 尝试 multimask\_output=True 并选择最佳结果

## 成本估算

CLORE.AI 市场常见费率（截至 2024 年）：

| GPU       | 小时费率    | 日费率     | 4 小时会话  |
| --------- | ------- | ------- | ------- |
| RTX 3060  | \~$0.03 | \~$0.70 | \~$0.12 |
| RTX 3090  | \~$0.06 | \~$1.50 | \~$0.25 |
| RTX 4090  | \~$0.10 | \~$2.30 | \~$0.40 |
| A100 40GB | \~$0.17 | \~$4.00 | \~$0.70 |
| A100 80GB | \~$0.25 | \~$6.00 | \~$1.00 |

*价格因提供商和需求而异。请查看* [*CLORE.AI 市场*](https://clore.ai/marketplace) *以获取当前费率。*

**节省费用：**

* 使用 **竞价** 可中断工作市场——约三分之一的服务器将现货价格定得低于按需价格（中位数约优惠 13%），其余则与按需价格持平
* 使用 **CLORE** 代币支付
* 比较不同提供商的价格

## 下一步

* Stable Diffusion 修复
* [ControlNet 指南](/guides/guides_v2-zh/tu-xiang-chu-li/controlnet-advanced.md)
* [Real-ESRGAN 超分辨率放大](/guides/guides_v2-zh/tu-xiang-chu-li/real-esrgan-upscaling.md)


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.clore.ai/guides/guides_v2-zh/tu-xiang-chu-li/segment-anything.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.