Qwen3.8-27B (Dense VLM, One Card)

Deploy Qwen3.8-27B on Clore.ai — Alibaba's dense 27B vision-language model that runs on a single RTX 4090 at Q4 and a single RTX 5090 at FP8

Status (August 2026): Qwen released Qwen3.8-27B on 14 August 2026 under Apache 2.0, with an official FP8 checkpoint a day later. Weights: Qwen/Qwen3.8-27B and Qwen/Qwen3.8-27B-FP8. Dense 27B, native vision-language, 262,144-token context extensible to 1M with YaRN, day-one support in vLLM, SGLang and llama.cpp.

Most of what shipped this summer cannot be rented. GLM-5.2 is 756GB in FP8. MiniMax M3 is 854GB. Kimi K3 is 1.5TB and does not fit on any machine listed on Clore.ai at any quantization. Qwen3.8-27B is the counterweight: 27B dense parameters, 15.4GB at Q4, one card, one container, done. It is the model this marketplace is actually shaped for — there were 677 servers with a 24GB-or-better card free at the last snapshot.

It is also not a stripped-down model. It handles images and video natively, exposes a reasoning_effort dial, and its hybrid attention stack keeps the KV cache small enough that long context on a consumer card is realistic rather than theoretical.

Key Specs

Property	Value
Parameters	27B (dense)
Architecture	64 layers, hybrid Gated DeltaNet + Gated Attention, MTP
Modality	Text, image, video in → text out
Native Context	262,144 tokens
Extended Context	1,000,000 tokens (YaRN)
License	Apache 2.0
Release Date	14 August 2026
Weights	BF16 55.6GB · FP8 30.9GB · Q4_K_S GGUF 15.4GB
Primary Tooling	vLLM, SGLang, llama.cpp, Ollama

Why this one

• It fits. Q4 on a single RTX 3090 or 4090 at $0.07–0.42/hr; FP8 on an RTX PRO 6000 or 2× 24GB cards

• Apache 2.0 — commercial use, fine-tuning and redistribution with no revenue clause, unlike Kimi K3 (custom licence with a revenue gate) or MiniMax M3 (community licence)

• Hybrid attention — 3 of every 4 layer blocks are Gated DeltaNet (linear attention), so the KV cache grows far slower with context than in a standard transformer. This is what makes 100K+ context usable on 24GB

• Vision built in — diagrams, documents, screenshots and hour-scale video, no separate VLM to deploy

• Thinking control — reasoning_effort at low / medium / xhigh, plus preserve_thinking to carry reasoning across turns for agents

---

Requirements

One card is the whole point. A single RTX 4090 at Q4 gives you a frontier-class vision-language model for roughly the price of a cup of coffee per day. Step up to an RTX 5090 or an RTX PRO 6000 only if you want FP8 quality or a very long context.

	Q4_K_S GGUF	Q8 / FP8	BF16
Weights	15.4GB	~31GB	55.6GB
GPU	1× RTX 3090/4090 24GB	1× RTX PRO 6000 96GB, or 2× RTX 5090	2× RTX PRO 6000, or 4× RTX 5090
Practical context	~64K	~200K	full 262K
System RAM	32GB	64GB	96GB
Disk	25GB	45GB	80GB
CUDA	12.8+	12.8+	12.8+

A single RTX 5090 (32GB) is not enough for FP8. The weights alone are 30.9GB. You need headroom for the KV cache and activations, so FP8 wants either a 96GB card or two GPUs. On one 5090, run Q6 or Q4 — see CUDA & PyTorch Compatibility for the right base image on Blackwell.

---

Option A — Ollama (fastest path)

ollama pull qwen3.8:27b
ollama run qwen3.8:27b

# OpenAI-compatible endpoint
ollama serve &
curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3.8:27b",
    "messages": [{"role": "user", "content": "Refactor this handler to use context.Context and add retries."}],
    "temperature": 0.7
  }'

The default tag is Q4_K_M (~16GB). Use qwen3.8:27b-q8_0 if you rented a 96GB card.

---

Option B — vLLM (production)

Single 24GB card, community INT4 AWQ checkpoint:

# docker-compose.yml
services:
  vllm:
    image: vllm/vllm-openai:latest
    ports: ["8000:8000"]
    volumes: [hf_cache:/root/.cache/huggingface]
    command: >
      --model cyankiwi/Qwen3.8-27B-AWQ-INT4
      --max-model-len 65536
      --gpu-memory-utilization 0.92
      --served-model-name qwen3.8-27b
      --enable-auto-tool-choice
      --tool-call-parser hermes
    deploy:
      resources:
        reservations:
          devices: [{driver: nvidia, count: 1, capabilities: [gpu]}]
    shm_size: "8gb"
volumes:
  hf_cache:

Qwen ships only BF16 and FP8 officially. The 4-bit checkpoints are community builds — cyankiwi/Qwen3.8-27B-AWQ-INT4 and RedHatAI/Qwen3.8-27B-INT4 are the most downloaded, and unsloth/Qwen3.8-27B-NVFP4 targets Blackwell cards specifically.

FP8 across two GPUs:

vllm serve Qwen/Qwen3.8-27B-FP8 \
  --tensor-parallel-size 2 \
  --max-model-len 262144 \
  --gpu-memory-utilization 0.90 \
  --enable-chunked-prefill

Video input needs one extra flag:

vllm serve Qwen/Qwen3.8-27B --media-io-kwargs '{"video": {"num_frames": -1}}'

Push past the native window with YaRN:

VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 vllm serve Qwen/Qwen3.8-27B \
  --hf-overrides '{"text_config": {"rope_parameters": {"rope_type": "yarn", "factor": 4.0, "original_max_position_embeddings": 262144}}}' \
  --max-model-len 1000000

---

Option C — llama.cpp / GGUF (24GB cards)

# Q4_K_S — 15.4GB, comfortable on a 24GB card
huggingface-cli download unsloth/Qwen3.8-27B-GGUF \
  --include "*UD-Q4_K_S*" --local-dir /workspace/qwen38

llama-server -m /workspace/qwen38/*UD-Q4_K_S*.gguf \
  --host 0.0.0.0 --port 8080 \
  -ngl 999 -c 65536 --flash-attn

Smaller quants from the same repo, for tighter cards: UD-Q3_K_XL 13.1GB, UD-Q2_K_XL 9.8GB, UD-IQ2_S 8.4GB.

---

Reasoning effort

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="none")

resp = client.chat.completions.create(
    model="qwen3.8-27b",
    messages=[{"role": "user", "content": "Plan the migration in steps."}],
    extra_body={"reasoning_effort": "xhigh"},   # xhigh (default) | medium | low
)

Drop to low for classification, extraction and routing — the quality difference is small there and the token bill is a fraction. Keep xhigh for agents and multi-step code work.

---

Clore.ai GPU Recommendations

Setup	VRAM	Mode	Context	Clore.ai cost
1× RTX 3090	24GB	Q4 GGUF	~64K	$0.07–0.21/hr
1× RTX 4090	24GB	Q4 AWQ	~64K	$0.14–0.42/hr
1× RTX 5090	32GB	Q6 GGUF	~96K	$0.25–0.77/hr
2× RTX 5090	64GB	FP8, TP=2	~200K	$0.50–1.54/hr
1× RTX PRO 6000	96GB	FP8	full 262K	$0.92–1.38/hr
4× RTX 5090	128GB	BF16, TP=4	full 262K	$1.00–3.08/hr

Best value: a single RTX 4090 from $0.14/hr at Q4. If the whole-rig price looks high, remember that most multi-GPU rigs allow partial rental — you can take one card out of an eight-card box.

---

Benchmarks

Numbers below are Qwen's own, from the 14 August 2026 model card. Independent reproductions were still landing at the time of writing.

Qwen evaluates the 27B against much larger models on coding and agentic suites (SWE-bench Pro, DeepSWE, QwenSWEBench) using the Claude Code harness at 256K context. The claim that matters for deployment is not a single score: it is that a dense 27B running on one consumer card lands in the same conversation as MoE models that need a rack. Treat the vendor table as a starting hypothesis and benchmark on your own task set — that is cheap here, since an hour of 4090 time costs about twenty cents.

---

Use Cases

• Single-GPU coding assistant — one container, one card, an OpenAI-compatible endpoint

• Document and screenshot understanding — native vision, no separate pipeline

• Video analysis — hour-scale video in a single pass with vLLM's frame flag

• Long-context RAG — 262K native, and the hybrid attention keeps the KV cache affordable

• Agent workers — preserve_thinking keeps reasoning coherent across tool calls

• On-prem and air-gapped — Apache 2.0, nothing phones home

• LoRA fine-tuning — 27B dense is friendly to Unsloth and LLaMA-Factory on a single 24GB card

---

Troubleshooting

Issue	Fix
no kernel image is available on a 5090	Old CUDA build — use a cu128 image, see compatibility
OOM at FP8 on one RTX 5090	Expected: 30.9GB of weights on a 32GB card. Use Q6/Q4, or TP=2
OOM at long context on 24GB	Lower --max-model-len to 32768, or drop to UD-Q3_K_XL
YaRN rejected	Needs a recent vLLM/SGLang; pass it through --hf-overrides, not as a standalone flag
Video input ignored	Add --media-io-kwargs '{"video": {"num_frames": -1}}' (vLLM only)
Tool calls dropped	--enable-auto-tool-choice --tool-call-parser hermes
Quality drops past ~600K	YaRN extends positions, not understanding — keep critical content near the end of the prompt

---

Next Steps

• Predecessor: Qwen3.6-27B — the April dense 27B this replaces

• Multimodal sibling: Qwen3.5-Omni — adds audio in and speech out

• Step up: GLM-5.2 or Mistral Small 4 when 27B is not enough

• Serving: vLLM · SGLang · Ollama

• Fine-tune it: Unsloth

Links

• Qwen3.8-27B on Hugging Face · FP8 · GGUF

• vLLM recipe · SGLang cookbook

• Qwen blog

• Rent a GPU: RTX 4090 · RTX 5090 · Marketplace