> For the complete documentation index, see [llms.txt](https://docs.clore.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.clore.ai/guides/getting-started/cuda-pytorch-compatibility.md). # CUDA & PyTorch Compatibility Pick a CUDA build that matches the card you rented — the one setting that breaks more Clore.ai deployments than anything else {% hint style="danger" %} **If you rent an RTX 50-series card and run a CUDA 12.4 image, nothing will work.** You will get `no kernel image is available for execution on the device`. Blackwell needs **CUDA 12.8 or newer** and **PyTorch 2.7 or newer**. This is the single most common failed deployment on Clore.ai, and it is a one-line fix. {% endhint %} Every guide in this collection ships a base image. If you change it, or if you install PyTorch yourself, the CUDA build has to match the architecture of the GPU you rented. Clore.ai is a marketplace, not a uniform cluster: the fleet spans Pascal mining cards from 2017 to Blackwell workstation cards from 2025, and no single image covers all of them. ## The matrix | Architecture | Compute cap. | Cards on Clore.ai | Use this | | ------------- | --------------- | ---------------------------------------------------- | --------------------------------------------------- | | **Blackwell** | sm\_120 | RTX 5090, 5080, 5070 Ti, 5070, 5060 Ti, RTX PRO 6000 | PyTorch **≥ 2.7** on **cu128** or **cu13x** | | **Ada** | sm\_89 | RTX 4090, 4080, 4070 Ti, 4060 Ti | PyTorch ≥ 2.1 on cu121 … cu13x (cu128 recommended) | | **Ampere** | sm\_86 / sm\_80 | RTX 3090, 3080, 3070, 3060, A6000, A2000, CMP 90HX | PyTorch ≥ 1.13 on cu118 … cu13x (cu128 recommended) | | **Turing** | sm\_75 | RTX 2080 Ti, GTX 1660, CMP 30HX/40HX/50HX | Anything cu118 … cu13x | | **Volta** | sm\_70 | Tesla V100 (16GB and 32GB SXM2) | PyTorch **≤ 2.10** on cu128, or any **cu126** build | | **Pascal** | sm\_61 | P104-100 and similar mining cards | PyTorch **≤ 2.7** on **cu124** or older | Two deprecations catch people out: * **cu128 builds from PyTorch 2.8 dropped Maxwell and Pascal** (sm\_50, sm\_60). * **PyTorch 2.11 dropped Volta from its cu128 and cu129 binaries** (cuDNN 9.15.1 is not Volta-compatible). V100 renters should pin `2.10.0-cuda12.8` or use a cu126 build. * **CUDA 13.0 itself starts at Turing** (sm\_75). Any `cuda13x` image excludes Volta and Pascal by construction. ## Base images that work ```bash # Default for anything from Ampere to Blackwell — this is what the guides use pytorch/pytorch:2.11.0-cuda12.8-cudnn9-devel # build tools, compilers, nvcc pytorch/pytorch:2.11.0-cuda12.8-cudnn9-runtime # smaller, no nvcc # Newest, Turing and up only (no V100, no Pascal) pytorch/pytorch:2.13.0-cuda13.2-cudnn9-devel # Tesla V100 (sm_70) pytorch/pytorch:2.10.0-cuda12.8-cudnn9-devel # Pascal mining cards (sm_61) pytorch/pytorch:2.5.1-cuda12.4-cudnn9-devel ``` Installing PyTorch into your own image: ```bash # Ampere / Ada / Blackwell pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128 # Volta (V100) pip install "torch<=2.10" torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126 ``` ## Check what you actually rented ```bash # Driver and the CUDA version it exposes nvidia-smi # Compute capability of the card, straight from torch python -c "import torch; print(torch.cuda.get_device_name(0), torch.cuda.get_device_capability(0))" # RTX 5090 -> (12, 0) RTX 4090 -> (8, 9) RTX 3090 -> (8, 6) V100 -> (7, 0) # Which architectures this build actually has kernels for python -c "import torch; print(torch.cuda.get_arch_list())" ``` If `get_device_capability` returns `(12, 0)` and `get_arch_list()` has no `sm_120`, you are on the wrong image. Nothing else you change will help. {% hint style="info" %} **Filter before you rent.** The marketplace exposes each host's CUDA version. About **65% of Clore.ai GPUs currently sit on a 13.x driver** and 35% on 12.x, so a `cu128` image is the safest default: CUDA minor-version compatibility lets it run on both. {% endhint %} ## Error messages, decoded | What you see | What it means | Fix | | -------------------------------------------------------------- | ----------------------------------------------------- | ---------------------------------------------------------------------- | | `no kernel image is available for execution on the device` | The build has no kernels for your card's architecture | Match the image to the matrix above | | `CUDA error: no kernel image ... sm_120` | Blackwell card, pre-2.7 or cu124 build | Move to `cu128` + PyTorch ≥ 2.7 | | `CUDA driver version is insufficient for CUDA runtime version` | Image needs a newer driver than the host has | Pick a lower CUDA image, or rent a host on a newer driver | | `torch.cuda.is_available() == False` | Container started without GPU access | Redeploy; make sure the order actually attached a GPU | | `undefined symbol: __cudaPopCallConfiguration` | Mixed CUDA versions in one environment | Rebuild from one clean base image; do not `pip install` a second torch | | Works on the 4090, fails on the 5090 | Classic architecture gap | The image is Ada-era; move to cu128 | ## Flash Attention and friends Compiled extensions carry the same constraint as PyTorch, and usually lag a release behind: ```bash # Build against the torch you already have, not a fresh one pip install flash-attn --no-build-isolation # vLLM and SGLang ship their own CUDA builds — use their images rather than # installing them into a PyTorch image, or you will end up with two torches docker pull vllm/vllm-openai:latest docker pull lmsysorg/sglang:latest ``` `vllm/vllm-openai:latest` and `lmsysorg/sglang:latest` both carry Blackwell kernels. If you pin an older tag for reproducibility, check that it postdates the RTX 50-series, otherwise you inherit exactly the problem this page is about. ## Next Steps * [Docker Images Catalog](/guides/getting-started/docker-images.md) — ready-to-deploy images per workload * [GPU Comparison](/guides/getting-started/gpu-comparison.md) — which card for which model * [GPU Pricing](/guides/getting-started/pricing.md) — live marketplace ranges * [Troubleshooting](/guides/getting-started/clore-troubleshooting.md) — the rest of the failure modes ### Links * [PyTorch previous versions and CUDA builds](https://pytorch.org/get-started/previous-versions/) * [pytorch/pytorch on Docker Hub](https://hub.docker.com/r/pytorch/pytorch/tags) * [CUDA GPU compute capabilities](https://developer.nvidia.com/cuda-gpus) * **Rent a GPU:** [RTX 5090 32GB](https://clore.ai/rent-5090.html) · [RTX 4090 24GB](https://clore.ai/rent-4090.html) · [Marketplace](https://clore.ai/marketplace) --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.clore.ai/guides/getting-started/cuda-pytorch-compatibility.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.