> For the complete documentation index, see [llms.txt](https://docs.clore.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.clore.ai/guides/guides_v2-zh/shi-jue-mo-xing/groundingdino.md).

# GroundingDINO

使用 GroundingDINO 通过文本描述检测任意对象

使用 GroundingDINO 通过文本描述检测任何物体。

{% hint style="success" %}
所有示例都可以在通过以下方式租用的 GPU 服务器上运行 [CLORE.AI 市场](https://clore.ai/marketplace).
{% endhint %}

{% hint style="info" %}
本指南中的所有示例都可以在通过以下方式租用的 GPU 服务器上运行 [CLORE.AI 市场](https://clore.ai/marketplace) 市场。
{% endhint %}

## 在 CLORE.AI 上租用

1. 访问 [CLORE.AI 市场](https://clore.ai/marketplace)
2. 按 GPU 类型、VRAM 和价格筛选
3. 选择 **按需** （固定费率）或 **竞价** （出价）
4. 配置你的订单：
   * 选择 Docker 镜像
   * 设置端口（SSH 用 TCP，Web UI 用 HTTP）
   * 如有需要，添加环境变量
   * 输入启动命令
5. 选择支付方式： **CLORE**, **BTC**，或 **USDT/USDC**
6. 创建订单并等待部署

### 访问你的服务器

* 在以下位置查找连接信息 **我的订单**
* Web 界面：使用 HTTP 端口 URL
* SSH： `ssh -p <port> root@<proxy-address>`

## 什么是 GroundingDINO？

IDEA-Research 的 GroundingDINO 支持：

* 使用文本提示进行零样本目标检测
* 无需训练即可检测任何物体
* 高精度边界框定位
* 与 SAM 结合实现自动分割

## 资源

* **GitHub：** [IDEA-Research/GroundingDINO](https://github.com/IDEA-Research/GroundingDINO)
* **论文：** [GroundingDINO 论文](https://arxiv.org/abs/2303.05499)
* **HuggingFace：** [IDEA-Research/grounding-dino](https://huggingface.co/IDEA-Research/grounding-dino-base)
* **演示：** [HuggingFace Space](https://huggingface.co/spaces/IDEA-Research/Grounding_DINO_Demo)

## 推荐硬件

| 组件  | 最低            | 推荐            | 最佳            |
| --- | ------------- | ------------- | ------------- |
| GPU | RTX 3060 12GB | RTX 4080 16GB | RTX 4090 24GB |
| 显存  | 6GB           | 12GB          | 16GB          |
| CPU | 4 核           | 8 核           | 16 核          |
| 内存  | 16GB          | 32GB          | 64GB          |
| 存储  | 20GB SSD      | 50GB NVMe     | 100GB NVMe    |
| 网络  | 100 Mbps      | 500 Mbps      | 1 Gbps        |

## 在 CLORE.AI 上快速部署

**Docker 镜像：**

```
pytorch/pytorch:2.11.0-cuda12.8-cudnn9-devel
```

**端口：**

```
22/tcp
7860/http
```

**命令：**

```bash
cd /workspace && \
git clone https://github.com/IDEA-Research/GroundingDINO.git && \
cd GroundingDINO && \
pip install -e . && \
python demo/gradio_demo.py
```

## 访问你的服务

部署后，找到你的 `http_pub` URL 在 **我的订单**:

1. 前往 **我的订单** 页面
2. 点击你的订单
3. 找到 `http_pub` URL（例如， `abc123.clorecloud.net`)

使用 `https://YOUR_HTTP_PUB_URL` 替代 `localhost` 在下面的示例中。

## 安装

```bash
git clone https://github.com/IDEA-Research/GroundingDINO.git
cd GroundingDINO
pip install -e .

# 下载权重
mkdir weights
cd weights
wget https://github.com/IDEA-Research/GroundingDINO/releases/download/v0.1.0-alpha/groundingdino_swint_ogc.pth
```

## 你可以创建什么

### 自动标注

* 为机器学习训练自动标注数据集
* 根据描述生成边界框
* 加快数据标注流程

### 视觉搜索

* 在图像数据库中查找特定物体
* 内容审核系统
* 零售中的商品识别

### 机器人与自动化

* 用于机械臂的目标定位
* 库存管理系统
* 质量控制检测

### 创意应用

* 从照片中自动裁剪主体
* 使用 SAM 生成目标掩码
* 内容感知图像编辑

### 分析

* 统计图像中的物体数量
* 从照片中跟踪库存
* 野生动物监测

## 基础用法

```python
from groundingdino.util.inference import load_model, load_image, predict, annotate
import cv2

# 加载模型
model = load_model(
    "groundingdino/config/GroundingDINO_SwinT_OGC.py",
    "weights/groundingdino_swint_ogc.pth"
)

# 加载图像
image_source, image = load_image("input.jpg")

# 检测物体
TEXT_PROMPT = "cat . dog . person"
BOX_THRESHOLD = 0.35
TEXT_THRESHOLD = 0.25

boxes, logits, phrases = predict(
    model=model,
    image=image,
    caption=TEXT_PROMPT,
    box_threshold=BOX_THRESHOLD,
    text_threshold=TEXT_THRESHOLD
)

# 标注图像
annotated_frame = annotate(
    image_source=image_source,
    boxes=boxes,
    logits=logits,
    phrases=phrases
)

cv2.imwrite("output.jpg", annotated_frame)
```

## GroundingDINO + SAM（Grounded-SAM）

结合检测与分割：

```python
import torch
import numpy as np
from groundingdino.util.inference import load_model, load_image, predict
from segment_anything import sam_model_registry, SamPredictor

# 加载 GroundingDINO
dino_model = load_model(
    "groundingdino/config/GroundingDINO_SwinT_OGC.py",
    "weights/groundingdino_swint_ogc.pth"
)

# 加载 SAM
sam = sam_model_registry["vit_h"](checkpoint="sam_vit_h_4b8939.pth")
sam.to(device="cuda")
sam_predictor = SamPredictor(sam)

# 加载图像
image_source, image = load_image("input.jpg")

# 使用 GroundingDINO 检测
boxes, logits, phrases = predict(
    model=dino_model,
    image=image,
    caption="person . car",
    box_threshold=0.35,
    text_threshold=0.25
)

# 使用 SAM 分割
sam_predictor.set_image(image_source)

# 将边界框转换为 SAM 格式
H, W = image_source.shape[:2]
boxes_xyxy = boxes * torch.tensor([W, H, W, H])

masks = []
for box in boxes_xyxy:
    mask, _, _ = sam_predictor.predict(
        box=box.numpy(),
        multimask_output=False
    )
    masks.append(mask)
```

## 批量处理

```python
import os
from groundingdino.util.inference import load_model, load_image, predict, annotate
import cv2

model = load_model(
    "groundingdino/config/GroundingDINO_SwinT_OGC.py",
    "weights/groundingdino_swint_ogc.pth"
)

input_dir = "./images"
output_dir = "./detected"
os.makedirs(output_dir, exist_ok=True)

TEXT_PROMPT = "product . price tag . barcode"

for filename in os.listdir(input_dir):
    if not filename.endswith(('.jpg', '.png')):
        continue

    image_path = os.path.join(input_dir, filename)
    image_source, image = load_image(image_path)

    boxes, logits, phrases = predict(
        model=model,
        image=image,
        caption=TEXT_PROMPT,
        box_threshold=0.3,
        text_threshold=0.25
    )

    annotated = annotate(image_source, boxes, logits, phrases)
    cv2.imwrite(os.path.join(output_dir, filename), annotated)

    print(f"{filename}: 找到 {len(boxes)} 个物体")
```

## 自定义检测流程

```python
from groundingdino.util.inference import load_model, load_image, predict
import json

model = load_model(
    "groundingdino/config/GroundingDINO_SwinT_OGC.py",
    "weights/groundingdino_swint_ogc.pth"
)

def detect_and_export(image_path, prompt, output_json):
    image_source, image = load_image(image_path)
    H, W = image_source.shape[:2]

    boxes, logits, phrases = predict(
        model=model,
        image=image,
        caption=prompt,
        box_threshold=0.35,
        text_threshold=0.25
    )

    # 转换为绝对坐标
    detections = []
    for box, logit, phrase in zip(boxes, logits, phrases):
        x1, y1, x2, y2 = box * torch.tensor([W, H, W, H])
        detections.append({
            "label": phrase,
            "confidence": float(logit),
            "bbox": {
                "x1": int(x1),
                "y1": int(y1),
                "x2": int(x2),
                "y2": int(y2)
            }
        })

    with open(output_json, "w") as f:
        json.dump(detections, f, indent=2)

    return detections

# 检测汽车和行人
results = detect_and_export(
    "street.jpg",
    "car . person . bicycle . traffic light",
    "detections.json"
)
```

## Gradio 界面

```python
import gradio as gr
import cv2
from groundingdino.util.inference import load_model, load_image, predict, annotate
import tempfile
import numpy as np

model = load_model(
    "groundingdino/config/GroundingDINO_SwinT_OGC.py",
    "weights/groundingdino_swint_ogc.pth"
)

def detect_objects(image, text_prompt, box_threshold, text_threshold):
    # 保存临时图像
    with tempfile.NamedTemporaryFile(suffix=".jpg", delete=False) as f:
        cv2.imwrite(f.name, cv2.cvtColor(np.array(image), cv2.COLOR_RGB2BGR))
        image_source, img = load_image(f.name)

    boxes, logits, phrases = predict(
        model=model,
        image=img,
        caption=text_prompt,
        box_threshold=box_threshold,
        text_threshold=text_threshold
    )

    annotated = annotate(image_source, boxes, logits, phrases)
    annotated_rgb = cv2.cvtColor(annotated, cv2.COLOR_BGR2RGB)

    return annotated_rgb, f"找到 {len(boxes)} 个物体：{', '.join(phrases)}"

demo = gr.Interface(
    fn=detect_objects,
    inputs=[
        gr.Image(type="pil", label="输入图像"),
        gr.Textbox(label="要检测的物体", value="person . car . dog", placeholder="object1 . object2 . object3"),
        gr.Slider(0.1, 0.9, value=0.35, label="边界框阈值"),
        gr.Slider(0.1, 0.9, value=0.25, label="文本阈值")
    ],
    outputs=[
        gr.Image(label="检测结果"),
        gr.Textbox(label="摘要")
    ],
    title="GroundingDINO - 开放集目标检测",
    description="通过文本描述来检测任何物体。运行在 CLORE.AI GPU 服务器上。"
)

demo.launch(server_name="0.0.0.0", server_port=7860)
```

## 性能

| 任务        | 分辨率       | GPU      | 速度    |
| --------- | --------- | -------- | ----- |
| 单张图像      | 800x600   | RTX 3090 | 120ms |
| 单张图像      | 800x600   | RTX 4090 | 80ms  |
| 单张图像      | 1920x1080 | RTX 4090 | 150ms |
| 批量（10张图片） | 800x600   | RTX 4090 | 600毫秒 |

## 常见问题与解决方案

### 检测准确率低

**问题：** 未检测到物体

**解决方案：**

* 将 `box_threshold` 调至 0.2-0.3
* 将 `text_threshold` 调至 0.15-0.2
* 使用更具体的物体描述
* 用 " . " 分隔物体，而不是逗号

```python

# 正确的提示格式
TEXT_PROMPT = "red car . person wearing hat . wooden chair"

# 错误的提示格式
TEXT_PROMPT = "red car, person wearing hat, wooden chair"
```

### 内存不足

**问题：** 大图像时 CUDA 内存不足

**解决方案：**

```python

# 在检测前缩小大图像
from PIL import Image

def resize_if_needed(image_path, max_size=1280):
    img = Image.open(image_path)
    if max(img.size) > max_size:
        ratio = max_size / max(img.size)
        new_size = (int(img.width * ratio), int(img.height * ratio))
        img = img.resize(new_size, Image.LANCZOS)
        img.save(image_path)
```

### 推理缓慢

**问题：** 检测耗时过长

**解决方案：**

* 使用更小的输入图像
* 批量处理多张图像
* 使用 FP16 推理
* 租用更快的 GPU（RTX 4090、A100）

### 误报

**问题：** 检测到了错误的物体

**解决方案：**

* 增大 `box_threshold` 调至 0.4-0.5
* 在提示中更具体一些
* 使用负向提示（在检测后过滤结果）

```python

# 过滤低置信度检测结果
filtered = [(b, l, p) for b, l, p in zip(boxes, logits, phrases) if l > 0.5]
```

## 故障排查

### 未检测到对象

* 使用更具体的文本描述
* 尝试不同的表述方式
* 降低置信度阈值

### 边界框不正确

* 在文本提示中更具体一些
* 使用 "." 分隔多个物体
* 检查图像质量

{% hint style="danger" %}
**内存不足**
{% endhint %}

* 降低图像分辨率
* 一次处理一张图像
* 使用更小的模型变体

### 推理缓慢

* 使用 TensorRT 加速
* 批量处理相近尺寸的图像
* 启用 FP16 推理

## 成本估算

CLORE.AI 市场常见费率（截至 2024 年）：

| GPU       | 小时费率    | 日费率     | 4 小时会话  |
| --------- | ------- | ------- | ------- |
| RTX 3060  | \~$0.03 | \~$0.70 | \~$0.12 |
| RTX 3090  | \~$0.06 | \~$1.50 | \~$0.25 |
| RTX 4090  | \~$0.10 | \~$2.30 | \~$0.40 |
| A100 40GB | \~$0.17 | \~$4.00 | \~$0.70 |
| A100 80GB | \~$0.25 | \~$6.00 | \~$1.00 |

*价格因提供商和需求而异。请查看* [*CLORE.AI 市场*](https://clore.ai/marketplace) *以获取当前费率。*

**节省费用：**

* 使用 **竞价** 可中断工作市场——约三分之一的服务器将现货价格定得低于按需价格（中位数约优惠 13%），其余则与按需价格持平
* 使用 **CLORE** 代币支付
* 比较不同提供商的价格

## 下一步

* [SAM2](/guides/guides_v2-zh/shi-jue-mo-xing/sam2-video.md) - 分割检测到的对象
* [Florence-2](/guides/guides_v2-zh/shi-jue-mo-xing/florence2.md) - 更多视觉任务
* [YOLO](/guides/guides_v2-zh/ji-suan-ji-shi-jue/yolov8-detection.md) - 针对已知类别的更快检测


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.clore.ai/guides/guides_v2-zh/shi-jue-mo-xing/groundingdino.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.