> For the complete documentation index, see [llms.txt](https://docs.clore.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.clore.ai/guides/language-models/qwen38-27b.md).

# Qwen3.8-27B (Dense VLM, One Card)

Deploy Qwen3.8-27B on Clore.ai — Alibaba's dense 27B vision-language model that runs on a single RTX 4090 at Q4 and a single RTX 5090 at FP8

{% hint style="info" %}
**Status (August 2026):** Qwen released **Qwen3.8-27B** on **14 August 2026** under **Apache 2.0**, with an official FP8 checkpoint a day later. Weights: [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) and [Qwen/Qwen3.8-27B-FP8](https://huggingface.co/Qwen/Qwen3.8-27B-FP8). Dense 27B, **native vision-language**, **262,144-token context** extensible to **1M with YaRN**, day-one support in vLLM, SGLang and llama.cpp.
{% endhint %}

Most of what shipped this summer cannot be rented. GLM-5.2 is 756GB in FP8. MiniMax M3 is 854GB. Kimi K3 is 1.5TB and does not fit on any machine listed on Clore.ai at any quantization. Qwen3.8-27B is the counterweight: **27B dense parameters, 15.4GB at Q4, one card, one container, done.** It is the model this marketplace is actually shaped for — there were 677 servers with a 24GB-or-better card free at the last snapshot.

It is also not a stripped-down model. It handles images and video natively, exposes a `reasoning_effort` dial, and its hybrid attention stack keeps the KV cache small enough that long context on a consumer card is realistic rather than theoretical.

### Key Specs

| Property         | Value                                                   |
| ---------------- | ------------------------------------------------------- |
| Parameters       | 27B (dense)                                             |
| Architecture     | 64 layers, hybrid Gated DeltaNet + Gated Attention, MTP |
| Modality         | Text, image, video in → text out                        |
| Native Context   | 262,144 tokens                                          |
| Extended Context | 1,000,000 tokens (YaRN)                                 |
| License          | Apache 2.0                                              |
| Release Date     | 14 August 2026                                          |
| Weights          | BF16 55.6GB · FP8 30.9GB · Q4\_K\_S GGUF 15.4GB         |
| Primary Tooling  | vLLM, SGLang, llama.cpp, Ollama                         |

### Why this one

* **It fits.** Q4 on a single RTX 3090 or 4090 at **$0.07–0.42/hr**; FP8 on an RTX PRO 6000 or 2× 24GB cards
* **Apache 2.0** — commercial use, fine-tuning and redistribution with no revenue clause, unlike Kimi K3 (custom licence with a revenue gate) or MiniMax M3 (community licence)
* **Hybrid attention** — 3 of every 4 layer blocks are Gated DeltaNet (linear attention), so the KV cache grows far slower with context than in a standard transformer. This is what makes 100K+ context usable on 24GB
* **Vision built in** — diagrams, documents, screenshots and hour-scale video, no separate VLM to deploy
* **Thinking control** — `reasoning_effort` at `low` / `medium` / `xhigh`, plus `preserve_thinking` to carry reasoning across turns for agents

***

## Requirements

{% hint style="success" %}
**One card is the whole point.** A single RTX 4090 at Q4 gives you a frontier-class vision-language model for roughly the price of a cup of coffee per day. Step up to an RTX 5090 or an RTX PRO 6000 only if you want FP8 quality or a very long context.
{% endhint %}

|                   | Q4\_K\_S GGUF         | Q8 / FP8                             | BF16                            |
| ----------------- | --------------------- | ------------------------------------ | ------------------------------- |
| Weights           | 15.4GB                | \~31GB                               | 55.6GB                          |
| GPU               | 1× RTX 3090/4090 24GB | 1× RTX PRO 6000 96GB, or 2× RTX 5090 | 2× RTX PRO 6000, or 4× RTX 5090 |
| Practical context | \~64K                 | \~200K                               | full 262K                       |
| System RAM        | 32GB                  | 64GB                                 | 96GB                            |
| Disk              | 25GB                  | 45GB                                 | 80GB                            |
| CUDA              | 12.8+                 | 12.8+                                | 12.8+                           |

{% hint style="warning" %}
**A single RTX 5090 (32GB) is not enough for FP8.** The weights alone are 30.9GB. You need headroom for the KV cache and activations, so FP8 wants either a 96GB card or two GPUs. On one 5090, run Q6 or Q4 — see [CUDA & PyTorch Compatibility](/guides/getting-started/cuda-pytorch-compatibility.md) for the right base image on Blackwell.
{% endhint %}

***

## Option A — Ollama (fastest path)

```bash
ollama pull qwen3.8:27b
ollama run qwen3.8:27b

# OpenAI-compatible endpoint
ollama serve &
curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3.8:27b",
    "messages": [{"role": "user", "content": "Refactor this handler to use context.Context and add retries."}],
    "temperature": 0.7
  }'
```

The default tag is Q4\_K\_M (\~16GB). Use `qwen3.8:27b-q8_0` if you rented a 96GB card.

***

## Option B — vLLM (production)

Single 24GB card, community INT4 AWQ checkpoint:

```yaml
# docker-compose.yml
services:
  vllm:
    image: vllm/vllm-openai:latest
    ports: ["8000:8000"]
    volumes: [hf_cache:/root/.cache/huggingface]
    command: >
      --model cyankiwi/Qwen3.8-27B-AWQ-INT4
      --max-model-len 65536
      --gpu-memory-utilization 0.92
      --served-model-name qwen3.8-27b
      --enable-auto-tool-choice
      --tool-call-parser hermes
    deploy:
      resources:
        reservations:
          devices: [{driver: nvidia, count: 1, capabilities: [gpu]}]
    shm_size: "8gb"
volumes:
  hf_cache:
```

Qwen ships only BF16 and FP8 officially. The 4-bit checkpoints are community builds — `cyankiwi/Qwen3.8-27B-AWQ-INT4` and `RedHatAI/Qwen3.8-27B-INT4` are the most downloaded, and `unsloth/Qwen3.8-27B-NVFP4` targets Blackwell cards specifically.

FP8 across two GPUs:

```bash
vllm serve Qwen/Qwen3.8-27B-FP8 \
  --tensor-parallel-size 2 \
  --max-model-len 262144 \
  --gpu-memory-utilization 0.90 \
  --enable-chunked-prefill
```

Video input needs one extra flag:

```bash
vllm serve Qwen/Qwen3.8-27B --media-io-kwargs '{"video": {"num_frames": -1}}'
```

Push past the native window with YaRN:

```bash
VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 vllm serve Qwen/Qwen3.8-27B \
  --hf-overrides '{"text_config": {"rope_parameters": {"rope_type": "yarn", "factor": 4.0, "original_max_position_embeddings": 262144}}}' \
  --max-model-len 1000000
```

***

## Option C — llama.cpp / GGUF (24GB cards)

```bash
# Q4_K_S — 15.4GB, comfortable on a 24GB card
huggingface-cli download unsloth/Qwen3.8-27B-GGUF \
  --include "*UD-Q4_K_S*" --local-dir /workspace/qwen38

llama-server -m /workspace/qwen38/*UD-Q4_K_S*.gguf \
  --host 0.0.0.0 --port 8080 \
  -ngl 999 -c 65536 --flash-attn
```

Smaller quants from the same repo, for tighter cards: `UD-Q3_K_XL` 13.1GB, `UD-Q2_K_XL` 9.8GB, `UD-IQ2_S` 8.4GB.

***

## Reasoning effort

```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="none")

resp = client.chat.completions.create(
    model="qwen3.8-27b",
    messages=[{"role": "user", "content": "Plan the migration in steps."}],
    extra_body={"reasoning_effort": "xhigh"},   # xhigh (default) | medium | low
)
```

Drop to `low` for classification, extraction and routing — the quality difference is small there and the token bill is a fraction. Keep `xhigh` for agents and multi-step code work.

***

## Clore.ai GPU Recommendations

| Setup           | VRAM  | Mode       | Context   | Clore.ai cost     |
| --------------- | ----- | ---------- | --------- | ----------------- |
| **1× RTX 3090** | 24GB  | Q4 GGUF    | \~64K     | **$0.07–0.21/hr** |
| **1× RTX 4090** | 24GB  | Q4 AWQ     | \~64K     | **$0.14–0.42/hr** |
| 1× RTX 5090     | 32GB  | Q6 GGUF    | \~96K     | $0.25–0.77/hr     |
| 2× RTX 5090     | 64GB  | FP8, TP=2  | \~200K    | $0.50–1.54/hr     |
| 1× RTX PRO 6000 | 96GB  | FP8        | full 262K | $0.92–1.38/hr     |
| 4× RTX 5090     | 128GB | BF16, TP=4 | full 262K | $1.00–3.08/hr     |

{% hint style="success" %}
**Best value:** a single [RTX 4090 from $0.14/hr](https://clore.ai/rent-4090.html) at Q4. If the whole-rig price looks high, remember that most multi-GPU rigs allow [partial rental](/guides/getting-started/partial-gpu-rental.md) — you can take one card out of an eight-card box.
{% endhint %}

***

## Benchmarks

{% hint style="warning" %}
Numbers below are Qwen's own, from the 14 August 2026 model card. Independent reproductions were still landing at the time of writing.
{% endhint %}

Qwen evaluates the 27B against much larger models on coding and agentic suites (SWE-bench Pro, DeepSWE, QwenSWEBench) using the Claude Code harness at 256K context. The claim that matters for deployment is not a single score: it is that a dense 27B running on one consumer card lands in the same conversation as MoE models that need a rack. Treat the vendor table as a starting hypothesis and benchmark on your own task set — that is cheap here, since an hour of 4090 time costs about twenty cents.

***

## Use Cases

* **Single-GPU coding assistant** — one container, one card, an OpenAI-compatible endpoint
* **Document and screenshot understanding** — native vision, no separate pipeline
* **Video analysis** — hour-scale video in a single pass with vLLM's frame flag
* **Long-context RAG** — 262K native, and the hybrid attention keeps the KV cache affordable
* **Agent workers** — `preserve_thinking` keeps reasoning coherent across tool calls
* **On-prem and air-gapped** — Apache 2.0, nothing phones home
* **LoRA fine-tuning** — 27B dense is friendly to [Unsloth](/guides/training/unsloth-finetune.md) and [LLaMA-Factory](/guides/training/llama-factory.md) on a single 24GB card

***

## Troubleshooting

| Issue                                    | Fix                                                                                                            |
| ---------------------------------------- | -------------------------------------------------------------------------------------------------------------- |
| `no kernel image is available` on a 5090 | Old CUDA build — use a cu128 image, see [compatibility](/guides/getting-started/cuda-pytorch-compatibility.md) |
| OOM at FP8 on one RTX 5090               | Expected: 30.9GB of weights on a 32GB card. Use Q6/Q4, or TP=2                                                 |
| OOM at long context on 24GB              | Lower `--max-model-len` to 32768, or drop to `UD-Q3_K_XL`                                                      |
| YaRN rejected                            | Needs a recent vLLM/SGLang; pass it through `--hf-overrides`, not as a standalone flag                         |
| Video input ignored                      | Add `--media-io-kwargs '{"video": {"num_frames": -1}}'` (vLLM only)                                            |
| Tool calls dropped                       | `--enable-auto-tool-choice --tool-call-parser hermes`                                                          |
| Quality drops past \~600K                | YaRN extends positions, not understanding — keep critical content near the end of the prompt                   |

***

## Next Steps

* **Predecessor:** [Qwen3.6-27B](/guides/language-models/qwen36-27b.md) — the April dense 27B this replaces
* **Multimodal sibling:** [Qwen3.5-Omni](/guides/language-models/qwen35-omni.md) — adds audio in and speech out
* **Step up:** [GLM-5.2](/guides/language-models/glm-5-2.md) or [Mistral Small 4](/guides/language-models/mistral-small4.md) when 27B is not enough
* **Serving:** [vLLM](/guides/language-models/vllm.md) · [SGLang](/guides/language-models/sglang.md) · [Ollama](/guides/language-models/ollama.md)
* **Fine-tune it:** [Unsloth](/guides/training/unsloth-finetune.md)

### Links

* [Qwen3.8-27B on Hugging Face](https://huggingface.co/Qwen/Qwen3.8-27B) · [FP8](https://huggingface.co/Qwen/Qwen3.8-27B-FP8) · [GGUF](https://huggingface.co/unsloth/Qwen3.8-27B-GGUF)
* [vLLM recipe](https://recipes.vllm.ai/Qwen/Qwen3.8-27B) · [SGLang cookbook](https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.8-27B)
* [Qwen blog](https://qwenlm.github.io/)
* **Rent a GPU:** [RTX 4090](https://clore.ai/rent-4090.html) · [RTX 5090](https://clore.ai/rent-5090.html) · [Marketplace](https://clore.ai/marketplace)


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.clore.ai/guides/language-models/qwen38-27b.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.