> For the complete documentation index, see [llms.txt](https://docs.clore.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.clore.ai/guides/guides_v2-zh/yu-yan-mo-xing/codellama.md).

# CodeLlama

在 Clore.ai 上使用 CodeLlama 生成、补全并解释代码

{% hint style="info" %}
**更新的替代方案！** 对于编码任务，可考虑 [**Qwen2.5-Coder**](/guides/guides_v2-zh/yu-yan-mo-xing/qwen25.md) (32B，最先进的代码生成) 或 [**DeepSeek-R1**](/guides/guides_v2-zh/yu-yan-mo-xing/deepseek-r1.md) (推理 + 编码)。CodeLlama 仍然适用于轻量级部署。
{% endhint %}

使用 Meta 的 CodeLlama 生成、补全并解释代码。

{% hint style="success" %}
所有示例都可以在通过以下方式租用的 GPU 服务器上运行 [CLORE.AI 市场](https://clore.ai/marketplace).
{% endhint %}

## 在 CLORE.AI 上租用

1. 访问 [CLORE.AI 市场](https://clore.ai/marketplace)
2. 按 GPU 类型、VRAM 和价格筛选
3. 选择 **按需** （固定费率）或 **竞价** （出价）
4. 配置你的订单：
   * 选择 Docker 镜像
   * 设置端口（SSH 用 TCP，Web UI 用 HTTP）
   * 如有需要，添加环境变量
   * 输入启动命令
5. 选择支付方式： **CLORE**, **BTC**，或 **USDT/USDC**
6. 创建订单并等待部署

### 访问你的服务器

* 在以下位置查找连接信息 **我的订单**
* Web 界面：使用 HTTP 端口 URL
* SSH： `ssh -p <port> root@<proxy-address>`

## 模型变体

| 模型            | 大小  | 显存    | 最适合  |
| ------------- | --- | ----- | ---- |
| CodeLlama-7B  | 7B  | 8GB   | 快速补全 |
| CodeLlama-13B | 13B | 16GB  | 平衡   |
| CodeLlama-34B | 34B | 40GB  | 最佳质量 |
| CodeLlama-70B | 70B | 80GB+ | 最高质量 |

### 变体

* **基础版**：代码补全
* **指令版**：遵循指令
* **Python**：Python 专用

## 快速部署

**Docker 镜像：**

```
pytorch/pytorch:2.11.0-cuda12.8-cudnn9-runtime
```

**端口：**

```
22/tcp
8000/http
```

**命令：**

```bash
pip install vllm && \\
python -m vllm.entrypoints.openai.api_server \
    --model codellama/CodeLlama-7b-Instruct-hf \\
    --port 8000
```

## 访问你的服务

部署后，找到你的 `http_pub` URL 在 **我的订单**:

1. 前往 **我的订单** 页面
2. 点击你的订单
3. 找到 `http_pub` URL（例如， `abc123.clorecloud.net`)

使用 `https://YOUR_HTTP_PUB_URL` 替代 `localhost` 在下面的示例中。

## 安装

### 使用 Ollama

```bash

# 安装 Ollama
curl -fsSL https://ollama.com/install.sh | sh

# 运行 CodeLlama
ollama run codellama

# 运行 Python 变体
ollama run codellama:python
```

### 使用 Transformers

```bash
pip install transformers accelerate
```

## 代码补全

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "codellama/CodeLlama-7b-hf"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto"
)

# 代码补全
code = """
def fibonacci(n):
    '''计算第 n 个斐波那契数'''
"""

inputs = tokenizer(code, return_tensors="pt").to("cuda")

outputs = model.generate(
    **inputs,
    max_new_tokens=200,
    temperature=0.2,
    do_sample=True
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## 指令模型

用于遵循编码指令：

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "codellama/CodeLlama-7b-Instruct-hf"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto"
)

prompt = """[INST] 编写一个 Python 函数，它：
1. 接收一个数字列表
2. 移除重复项
3. 按降序排序
4. 返回前 5 个元素
[/INST]"""

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

outputs = model.generate(
    **inputs,
    max_new_tokens=500,
    temperature=0.2
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## 中间填充（FIM）

```python

# CodeLlama 支持用于代码插入的 FIM
prefix = """def calculate_area(shape, dimensions):
    if shape == "circle":
        radius = dimensions[0]
"""

suffix = """
    elif shape == "rectangle":
        length, width = dimensions
        return length * width
    return None
"""

# 为 FIM 使用特殊标记
prompt = f"<PRE> {prefix} <SUF>{suffix} <MID>"

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=100)
```

## Python 专用模型

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "codellama/CodeLlama-7b-Python-hf"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto"
)

# Python 专用补全
code = """
import pandas as pd
import numpy as np

def analyze_sales_data(df):
    '''分析销售数据并返回关键指标'''
"""

inputs = tokenizer(code, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=300)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## vLLM 服务器

```bash
python -m vllm.entrypoints.openai.api_server \
    --model codellama/CodeLlama-13b-Instruct-hf \\
    --dtype float16 \\
    --max-model-len 8192
```

### API 使用

```python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="x")

response = client.chat.completions.create(
    model="codellama/CodeLlama-13b-Instruct-hf",
    messages=[
        {"role": "user", "content": "为用户身份验证编写一个 FastAPI 端点"}
    ],
    temperature=0.2,
    max_tokens=1000
)

print(response.choices[0].message.content)
```

## 代码解释

```python
code_to_explain = """
def quicksort(arr):
    if len(arr) <= 1:
        return arr
    pivot = arr[len(arr) // 2]
    left = [x for x in arr if x < pivot]
    middle = [x for x in arr if x == pivot]
    right = [x for x in arr if x > pivot]
    return quicksort(left) + middle + quicksort(right)
"""

prompt = f"[INST] 逐步解释这段代码：\n\n{code_to_explain}\n[/INST]"
```

## 修复错误

```python
buggy_code = """
def reverse_string(s):
    result = ""
    for i in range(len(s)):
        result += s[i]
    return result
"""

prompt = f"""[INST] 找出并修复这段代码中的错误。该函数应反转一个字符串：

{buggy_code}
[/INST]"""
```

## 代码翻译

```python
python_code = """
def factorial(n):
    if n <= 1:
        return 1
    return n * factorial(n - 1)
"""

prompt = f"""[INST] 将这段 Python 代码转换为 JavaScript：

{python_code}
[/INST]"""
```

## Gradio 界面

```python
import gradio as gr
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "codellama/CodeLlama-7b-Instruct-hf"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto"
)

def generate_code(instruction, temperature, max_tokens):
    prompt = f"[INST] {instruction} [/INST]"
    inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

    outputs = model.generate(
        **inputs,
        max_new_tokens=max_tokens,
        temperature=temperature,
        do_sample=True
    )

    response = tokenizer.decode(outputs[0], skip_special_tokens=True)
    return response.split("[/INST]")[-1].strip()

demo = gr.Interface(
    fn=generate_code,
    inputs=[
        gr.Textbox(label="指令", lines=5, placeholder="编写一个 Python 函数，它..."),
        gr.Slider(0.1, 1.0, value=0.2, label="温度"),
        gr.Slider(100, 2000, value=500, step=100, label="最大 Token 数")
    ],
    outputs=gr.Code(language="python", label="生成的代码"),
    title="CodeLlama 代码生成器"
)

demo.launch(server_name="0.0.0.0", server_port=7860)
```

## 批量处理

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "codellama/CodeLlama-7b-Instruct-hf"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto"
)

tasks = [
    "编写一个函数来验证电子邮件地址",
    "创建一个用于管理购物车的类",
    "编写一个函数，从 URL 解析 JSON",
    "创建一个用于计时函数执行的装饰器",
    "编写一个函数来生成随机密码"
]

for task in tasks:
    prompt = f"[INST] {task} [/INST]"
    inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

    outputs = model.generate(
        **inputs,
        max_new_tokens=500,
        temperature=0.2
    )

    result = tokenizer.decode(outputs[0], skip_special_tokens=True)
    print(f"\n=== {task} ===")
    print(result.split("[/INST]")[-1].strip())
```

## 与 Continue（VSCode）配合使用

配置 Continue 扩展：

```json
{
  "models": [
    {
      "title": "CodeLlama",
      "provider": "ollama",
      "model": "codellama:7b-instruct"
    }
  ],
  "tabAutocompleteModel": {
    "title": "CodeLlama",
    "provider": "ollama",
    "model": "codellama:7b-code"
  }
}
```

## 性能

| 模型            | GPU      | 每秒 Token 数 |
| ------------- | -------- | ---------- |
| CodeLlama-7B  | RTX 3090 | \~90       |
| CodeLlama-7B  | RTX 4090 | \~130      |
| CodeLlama-13B | RTX 4090 | \~70       |
| CodeLlama-34B | A100     | \~50       |

## 故障排查

### 代码质量差

* 降低温度（0.1-0.3）
* 使用指令版变体
* 如有可能，使用更大的模型

### 输出不完整

* 增加 max\_new\_tokens
* 检查上下文长度

### 生成缓慢

* 使用 vLLM
* 量化模型
* 使用更小的变体

## 成本估算

CLORE.AI 市场常见费率（截至 2024 年）：

| GPU       | 小时费率    | 日费率     | 4 小时会话  |
| --------- | ------- | ------- | ------- |
| RTX 3060  | \~$0.03 | \~$0.70 | \~$0.12 |
| RTX 3090  | \~$0.06 | \~$1.50 | \~$0.25 |
| RTX 4090  | \~$0.10 | \~$2.30 | \~$0.40 |
| A100 40GB | \~$0.17 | \~$4.00 | \~$0.70 |
| A100 80GB | \~$0.25 | \~$6.00 | \~$1.00 |

*价格因提供商和需求而异。请查看* [*CLORE.AI 市场*](https://clore.ai/marketplace) *以获取当前费率。*

**节省费用：**

* 使用 **竞价** 可中断工作市场——约三分之一的服务器将现货价格定得低于按需价格（中位数约优惠 13%），其余则与按需价格持平
* 使用 **CLORE** 代币支付
* 比较不同提供商的价格

## 下一步

* Open Interpreter - 执行代码
* vLLM 推理 - 生产环境服务
* Mistral/Mixtral - 替代模型


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.clore.ai/guides/guides_v2-zh/yu-yan-mo-xing/codellama.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.