NVIDIA Nemotron 3 Ultra (550B Mamba-MoE)

Run NVIDIA Nemotron 3 Ultra, the 550B open Mamba-MoE hybrid, on Blackwell rigs from the Clore.ai marketplace

Status (August 2026): NVIDIA announced Nemotron 3 Ultra at Computex on 4 June 2026 and released it under the OpenMDW-1.1 licence — weights, training data, recipe and the reinforcement-learning environment. 550B total / 55B active, hybrid Mamba-2 + MoE + attention architecture with multi-token prediction, context up to 1M tokens. Weights: nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 and an official NVFP4 build.

Two things set this release apart from the Chinese open-weight wave. First, the licence is unusually complete: OpenMDW-1.1 covers the model, the data and the recipe, so the whole pipeline is reproducible rather than just the checkpoint. Second, the architecture is not a plain transformer — interleaved Mamba-2 state-space layers with MoE and select attention layers, which is why NVIDIA quotes throughput well above what a 550B dense-attention model would give.

For Clore.ai renters there is a neat fit: the official NVFP4 checkpoint is built for Blackwell, and the biggest rigs on the marketplace are 4× RTX PRO 6000 Blackwell (380GB).

Key Specs

Property	Value
Parameters	550B total, 55B active
Architecture	LatentMoE — Mamba-2 + MoE + attention hybrid, MTP, 512 experts (22 active)
Context	Up to 1,000,000 tokens
License	OpenMDW-1.1 (model, data, recipe, RL environment)
Release Date	4 June 2026
Weights	NVFP4 352GB · Q2_K_XL GGUF 202GB · Q3_K_M GGUF 274GB
Primary Tooling	vLLM, SGLang, TensorRT-LLM, llama.cpp

---

Requirements

Build	Size	Clore.ai rig
UD-IQ1_M	188GB	8× RTX 5090 (248GB)
UD-Q2_K_XL	202GB	8× RTX 5090 (248GB) — 47 servers at ≥242GB
UD-Q3_K_M	274GB	10× RTX 5090 (310GB) or 4× RTX PRO 6000 (380GB)
NVFP4 official	352GB	4× RTX PRO 6000 Blackwell (380GB) — 2 servers listed
BF16 official	~1.1TB	bare metal

NVFP4 requires Blackwell. The FP4 tensor cores exist on RTX 50-series and RTX PRO 6000, not on Ada or Ampere. On a 4090 or 3090 rig, use the GGUF builds instead. See CUDA & PyTorch Compatibility.

---

Deploy

NVFP4 on a Blackwell rig (vLLM):

vllm serve nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4 \
  --tensor-parallel-size 4 \
  --trust-remote-code \
  --max-model-len 262144 \
  --gpu-memory-utilization 0.90 \
  --enable-expert-parallel

GGUF across an RTX 5090 rig (llama.cpp):

huggingface-cli download unsloth/NVIDIA-Nemotron-3-Ultra-550B-A55B-GGUF \
  --include "*UD-Q2_K_XL*" --local-dir /workspace/nemotron

llama-server -m /workspace/nemotron/*UD-Q2_K_XL*-00001-of-*.gguf \
  --host 0.0.0.0 --port 8080 \
  -ngl 999 --split-mode layer -c 131072 --flash-attn --no-mmap

Reasoning is a chat-template flag: the model emits a reasoning trace before its answer, and you can turn that off for latency-sensitive calls.

Other prebuilt checkpoints: RedHatAI/...-FP8-dynamic and RedHatAI/...-quantized.w4a16.

---

Clore.ai GPU Recommendations

Setup	Total VRAM	Build	Clore.ai cost
8× RTX 5090	248GB	Q2_K_XL GGUF	~$2.00–3.50/hr
10× RTX 5090	310GB	Q3_K_M GGUF	~$5.00/hr
4× RTX PRO 6000 Blackwell	380GB	NVFP4 official	~$5.00/hr

---

Use Cases

• Long-running agents — NVIDIA built the RL environment for exactly this and shipped it with the weights

• Reproducible research — data and recipe are covered by the licence, so results can be rebuilt, not just replayed

• High-throughput reasoning — the Mamba hybrid decodes faster than a dense-attention model of comparable size

• High-stakes RAG — long context plus a reasoning trace you can inspect

• US-jurisdiction open weights — the strongest US-developed open-weight model of the summer, where that matters procurement-wise

---

Troubleshooting

Issue	Fix
NVFP4 fails to load	The rig is not Blackwell — use GGUF or an FP8 build
nemotron_h architecture unknown	Update vLLM/SGLang/llama.cpp; the hybrid Mamba layers need recent support
Reasoning traces in every reply	Turn the reasoning flag off in the chat template for short-latency calls
Throughput below expectations	Enable MTP-based speculative decoding; the model ships MTP layers
OOM on 8× 5090 at Q3	Q3_K_M is 274GB — you need 310GB+ of VRAM

---

Next Steps

• Smaller sibling: Nemotron 3 Super — 120B MoE, fits far smaller rigs

• Same weight class: GLM-5.2 · MiniMax M3 · DeepSeek V4

• Single-card alternative: Qwen3.8-27B

• Serving: TensorRT-LLM · vLLM

Links

• Nemotron 3 Ultra BF16 · NVFP4 · GGUF

• OpenMDW-1.1 licence

• Rent a GPU: RTX 5090 · Marketplace