Hardware / NVIDIA
Can an RTX 5090 run Llama 3.3 70B?
RTX 5090 has 32 GB VRAM. About 28.8 GB is treated as usable. 32 GB Blackwell flagship, from rows that report 32 GB on a single card. The title asks about Llama 3.3 70B because it is the largest catalog model still in range of this memory.
Manufacturer page for RTX 5090
Compatibility
| Model | Q4 | Q8 | FP16 | MXFP4 |
|---|---|---|---|---|
| Gemma 4 31B Gemma · 30.7B stored | 33.4 GB Over estimate No measured run | 62.1 GB Over estimate No measured run | — | |
| Qwen3.8 27B Qwen · 27B stored | 29.9 GB Tight estimate · KV at 8k No measured run | 55.3 GB Over estimate · KV at 8k No measured run | — | |
| Gemma 4 26B-A4B Gemma · 25.2B stored · 3.8B active | 27.5 GB Fits estimate No measured run | 51.1 GB Over estimate No measured run | — | |
| gpt-oss 20B gpt-oss · 20.9B stored · 3.6B active | 23.0 GB Fits estimate No measured run | 42.6 GB Over estimate No measured run | ||
| Qwen3.6 35B-A3B Qwen · 35B stored · 3B active | 22.1 GB Fits estimate · KV at 8k No measured run | 38.1 GB Over estimate · KV at 8k No measured run | 70.9 GB Over estimate · KV at 8k No measured run | — |
| Qwen3 30B-A3B Qwen · 30B stored · 3B active | 18.9 GB Fits estimate No measured run | 32.6 GB Over estimate No measured run | 60.8 GB Over estimate No measured run | — |
| Qwen3 14B Qwen · 14B stored | 9.2 GB Fits estimate No measured run | 15.6 GB Fits estimate No measured run | 28.8 GB Fits estimate No measured run | — |
| DeepSeek R1 Distill 14B DeepSeek · 14B stored | 9.2 GB Fits estimate No measured run | 15.6 GB Fits estimate No measured run | 28.8 GB Fits estimate No measured run | — |
| Gemma 3 12B Gemma · 12B stored | 8.0 GB Fits estimate No measured run | 13.5 GB Fits estimate No measured run | 24.8 GB Fits estimate No measured run | — |
| Gemma 4 12B Gemma · 12B stored | 8.0 GB Fits estimate No measured run | 13.4 GB Fits estimate No measured run | 24.6 GB Fits estimate No measured run | — |
| Llama 3 8B Llama · 8B stored | 6.6 GB Fits estimate · KV at 8k No measured run | 10.3 GB Fits estimate · KV at 8k No measured run | 17.8 GB Fits estimate · KV at 8k No measured run | — |
| Llama 3.1 8B Llama · 8B stored | 6.6 GB Fits estimate · KV at 8k No measured run | 10.3 GB Fits estimate · KV at 8k No measured run | 17.8 GB Fits estimate · KV at 8k No measured run | — |
| Qwen3 8B Qwen · 8B stored | 5.6 GB Fits estimate No measured run | 9.3 GB Fits estimate No measured run | 16.8 GB Fits estimate No measured run | — |
| Gemma 4 E4B Gemma · 8B stored | 5.6 GB Fits estimate No measured run | 9.3 GB Fits estimate No measured run | 16.8 GB Fits estimate No measured run | — |
| DeepSeek R1 Distill 8B DeepSeek · 8B stored | 6.6 GB Fits estimate · KV at 8k No measured run | 10.3 GB Fits estimate · KV at 8k No measured run | 17.8 GB Fits estimate · KV at 8k No measured run | — |
| Llama 2 7B Llama · 7B stored | 9.0 GB Fits estimate · KV at 8k No measured run | 12.2 GB Fits estimate · KV at 8k No measured run | 18.8 GB Fits estimate · KV at 8k No measured run | — |
| Llama 3.2 3B Llama · 3B stored | 3.4 GB Fits estimate · KV at 8k No measured run | 4.8 GB Fits estimate · KV at 8k No measured run | 7.6 GB Fits estimate · KV at 8k No measured run | — |
| DeepSeek R1 671B DeepSeek · 671B stored · 37B active | 408 GB Over estimate No measured run | 714 GB Over estimate No measured run | 1343 GB Over estimate No measured run | — |
| Llama 3.1 405B Llama · 405B stored | 250 GB Over estimate · KV at 8k No measured run | 435 GB Over estimate · KV at 8k No measured run | 815 GB Over estimate · KV at 8k No measured run | — |
| Qwen3 235B-A22B Qwen · 235B stored · 22B active | 143 GB Over estimate No measured run | 250 GB Over estimate No measured run | 471 GB Over estimate No measured run | — |
| gpt-oss 120B gpt-oss · 117B stored · 5.1B active | 71.6 GB Over estimate No measured run | 125 GB Over estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Over checkpoint No measured run |
| Llama 3 70B Llama · 70B stored | 45.7 GB Over estimate · KV at 8k No measured run | 77.6 GB Over estimate · KV at 8k No measured run | 143 GB Over estimate · KV at 8k No measured run | — |
| Llama 3.1 70B Llama · 70B stored | 45.7 GB Over estimate · KV at 8k No measured run | 77.6 GB Over estimate · KV at 8k No measured run | 143 GB Over estimate · KV at 8k No measured run | — |
| Llama 3.3 70B Llama · 70B stored | 45.7 GB Over estimate · KV at 8k No measured run | 77.6 GB Over estimate · KV at 8k No measured run | 143 GB Over estimate · KV at 8k No measured run | — |
| DeepSeek R1 Distill 70B DeepSeek · 70B stored | 45.7 GB Over estimate · KV at 8k No measured run | 77.6 GB Over estimate · KV at 8k No measured run | 143 GB Over estimate · KV at 8k No measured run | — |
Measured runs on this device
- Qwen3.8 27B · all-NVFP4 NInfer v3 pack of the stock model (kaushikvira/Qwen3.8-27B-nvfp4full-dflash2-NInfer-v3) · 400 tok/s · speculative · NInfer (ninfer-all fork) + DFlash2 drafter, 7 draft tokens, INT8 KV source
- Qwen3.8 27B · all-NVFP4 NInfer v3 pack of the stock model (kaushikvira/Qwen3.8-27B-nvfp4full-dflash2-NInfer-v3) · 326 tok/s · speculative · NInfer (ninfer-all fork) + DFlash2 drafter, 7 draft tokens, INT8 KV source
- gpt-oss 20B · reported as Q4_K_M · 324.2 tok/s · llama.cpp 0.7.0-dev source
- gpt-oss 20B · llama-bench tg128 (quant per result) · 313 tok/s · llama.cpp source
- gpt-oss 20B · MXFP4 · 282.51 tok/s · llama.cpp source
- Llama 3.2 3B · not stated · 271.4 tok/s · llama.cpp source
- Qwen3.6 35B-A3B · llama-bench tg128 (quant per result) · 258 tok/s · llama.cpp source
- Gemma 4 26B-A4B · llama-bench tg128 (quant per result) · 235 tok/s · llama.cpp source
- Gemma 4 26B-A4B · Q4_K_M (text-only build), f16 KV · 232.4 tok/s · llama-server b10448 source
- Qwen3.8 27B · all-NVFP4 NInfer v3 pack of the stock model (kaushikvira/Qwen3.8-27B-nvfp4full-dflash2-NInfer-v3) · 165 tok/s · speculative · NInfer (ninfer-all fork) + DFlash2 drafter, 7 draft tokens, INT8 KV source
- Gemma 4 12B · llama-bench tg128 (quant per result) · 142 tok/s · llama.cpp source
- Qwen3.8 27B · Q4_K_M, q4_0 KV · 136.7 tok/s · speculative · llama.cpp b10448, MTP n=2 (--spec-type draft-mtp) source
- Qwen3.8 27B · Ollama qwen3.8:27b q4 (17 GB on disk) · 124 tok/s · Ollama 0.32.13 source
- Qwen3.8 27B · Q4_K_M (EPYC 9534 host) · 75.2 tok/s · llama.cpp 0.7.0-dev (b11669) source
- Qwen3.8 27B · Q4_K_M · 75.19 tok/s · llama.cpp source
- Qwen3.8 27B · Q4_K_M, f16 KV · 73.6 tok/s · llama.cpp b10448, MTP off source
- Gemma 4 31B · Q4_K_M · 71.3 tok/s · llama.cpp b11658 (llama-bench) source
- Gemma 4 31B · llama-bench tg128 (quant per result) · 70.0 tok/s · llama.cpp source