Hardware / NVIDIA
Can an RTX 3090 run Llama 3.3 70B?
RTX 3090 has 24 GB VRAM. About 21.6 GB is treated as usable. 24 GB GDDR6X. The usual used flagship in local-model builds. The title asks about Llama 3.3 70B because it is the largest catalog model still in range of this memory.
Manufacturer page for RTX 3090
Compatibility
| Model | Q4 | Q8 | FP16 | MXFP4 |
|---|---|---|---|---|
| gpt-oss 120B gpt-oss · 117B stored · 5.1B active | 125 GB Over estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Over checkpoint No measured run | |
| Qwen3.6 35B-A3B Qwen · 35B stored · 3B active | 38.1 GB Over estimate · KV at 8k No measured run | 70.9 GB Over estimate · KV at 8k No measured run | — | |
| Gemma 4 31B Gemma · 30.7B stored | 33.4 GB Over estimate No measured run | 62.1 GB Over estimate No measured run | — | |
| Qwen3.8 27B Qwen · 27B stored | 29.9 GB Over estimate · KV at 8k No measured run | 55.3 GB Over estimate · KV at 8k No measured run | — | |
| gpt-oss 20B gpt-oss · 20.9B stored · 3.6B active | 23.0 GB Tight estimate No measured run | 42.6 GB Over estimate No measured run | ||
| Qwen3 30B-A3B Qwen · 30B stored · 3B active | 18.9 GB Fits estimate No measured run | 32.6 GB Over estimate No measured run | 60.8 GB Over estimate No measured run | — |
| Gemma 4 26B-A4B Gemma · 25.2B stored · 3.8B active | 16.0 GB Fits estimate No measured run | 27.5 GB Over estimate No measured run | 51.1 GB Over estimate No measured run | — |
| Qwen3 14B Qwen · 14B stored | 9.2 GB Fits estimate No measured run | 15.6 GB Fits estimate No measured run | 28.8 GB Over estimate No measured run | — |
| DeepSeek R1 Distill 14B DeepSeek · 14B stored | 9.2 GB Fits estimate No measured run | 15.6 GB Fits estimate No measured run | 28.8 GB Over estimate No measured run | — |
| Gemma 3 12B Gemma · 12B stored | 8.0 GB Fits estimate No measured run | 13.5 GB Fits estimate No measured run | 24.8 GB Over estimate No measured run | — |
| Gemma 4 12B Gemma · 12B stored | 8.0 GB Fits estimate No measured run | 13.4 GB Fits estimate No measured run | 24.6 GB Over estimate No measured run | — |
| Llama 3 8B Llama · 8B stored | 6.6 GB Fits estimate · KV at 8k No measured run | 10.3 GB Fits estimate · KV at 8k No measured run | 17.8 GB Fits estimate · KV at 8k No measured run | — |
| Llama 3.1 8B Llama · 8B stored | 6.6 GB Fits estimate · KV at 8k No measured run | 10.3 GB Fits estimate · KV at 8k No measured run | 17.8 GB Fits estimate · KV at 8k No measured run | — |
| Qwen3 8B Qwen · 8B stored | 5.6 GB Fits estimate No measured run | 9.3 GB Fits estimate No measured run | 16.8 GB Fits estimate No measured run | — |
| Gemma 4 E4B Gemma · 8B stored | 5.6 GB Fits estimate No measured run | 9.3 GB Fits estimate No measured run | 16.8 GB Fits estimate No measured run | — |
| DeepSeek R1 Distill 8B DeepSeek · 8B stored | 6.6 GB Fits estimate · KV at 8k No measured run | 10.3 GB Fits estimate · KV at 8k No measured run | 17.8 GB Fits estimate · KV at 8k No measured run | — |
| Llama 2 7B Llama · 7B stored | 9.0 GB Fits estimate · KV at 8k No measured run | 12.2 GB Fits estimate · KV at 8k No measured run | 18.8 GB Fits estimate · KV at 8k No measured run | — |
| Llama 3.2 3B Llama · 3B stored | 3.4 GB Fits estimate · KV at 8k No measured run | 4.8 GB Fits estimate · KV at 8k No measured run | 7.6 GB Fits estimate · KV at 8k No measured run | — |
| DeepSeek R1 671B DeepSeek · 671B stored · 37B active
| 408 GB Over estimate No measured run | 714 GB Over estimate No measured run | 1343 GB Over estimate No measured run | — |
| Llama 3.1 405B Llama · 405B stored | 250 GB Over estimate · KV at 8k No measured run | 435 GB Over estimate · KV at 8k No measured run | 815 GB Over estimate · KV at 8k No measured run | — |
| Qwen3 235B-A22B Qwen · 235B stored · 22B active | 143 GB Over estimate No measured run | 250 GB Over estimate No measured run | 471 GB Over estimate No measured run | — |
| Llama 3 70B Llama · 70B stored | 45.7 GB Over estimate · KV at 8k No measured run | 77.6 GB Over estimate · KV at 8k No measured run | 143 GB Over estimate · KV at 8k No measured run | — |
| Llama 3.1 70B Llama · 70B stored | 45.7 GB Over estimate · KV at 8k No measured run | 77.6 GB Over estimate · KV at 8k No measured run | 143 GB Over estimate · KV at 8k No measured run | — |
| Llama 3.3 70B Llama · 70B stored | 45.7 GB Over estimate · KV at 8k No measured run | 77.6 GB Over estimate · KV at 8k No measured run | 143 GB Over estimate · KV at 8k No measured run | — |
| DeepSeek R1 Distill 70B DeepSeek · 70B stored | 45.7 GB Over estimate · KV at 8k No measured run | 77.6 GB Over estimate · KV at 8k No measured run | 143 GB Over estimate · KV at 8k No measured run | — |
Measured runs on this device
- gpt-oss 20B · Q4_K_M (bartowski GGUF, MXFP4 experts) · 211.1 tok/s · llama.cpp CUDA b11804 source
- gpt-oss 20B · reported as Q4_K_M · 207.2 tok/s · llama.cpp 0.7.0-dev source
- Qwen3.6 35B-A3B · Q4_K_M · 172.4 tok/s · llama.cpp CUDA b11804 source
- Qwen3.6 35B-A3B · Q4_K_M · 162.24 tok/s · llama.cpp v0.7.0 llama-bench (CUDA), fully on GPU source
- gpt-oss 20B · MXFP4 · 161.77 tok/s · llama.cpp source
- Qwen3.6 35B-A3B · llama-bench tg128 (quant per result) · 146–153 tok/s · llama.cpp source
- Qwen3.8 27B · Q4_K_M (unsloth), greedy code prompt · 139.7 tok/s · speculative · OpenJet megakernel (MTP, 4 drafts) source
- Qwen3.6 35B-A3B · Q4 GGUF, all layers on GPU, ~90K ctx · 139.6 tok/s · llama.cpp CUDA source
- Qwen3.8 27B · AutoRound W4A16 · 129.8 tok/s · speculative · vLLM (patched) source
- Qwen3.8 27B · Q4_K_M (unsloth), reasoning mode · 78.0 tok/s · speculative · OpenJet megakernel (MTP) source
- Qwen3.6 35B-A3B · Q4_K_M, 20 layers of experts in system RAM (-ncmoe 20) · 73.59 tok/s · llama.cpp v0.7.0 llama-server + --moe-cache-mib 1866 source
- Qwen3.8 27B · Q4_K_M (unsloth), greedy code prompt · 72.5 tok/s · speculative · llama.cpp (MTP, 2 drafts) source
- Qwen3.8 27B · Unsloth UD-Q4_K_M (nextn/MTP head embedded), f16 KV · 56.37 tok/s · speculative · llama.cpp b11382 (Windows, CUDA 13), MTP n=2 source
- Qwen3.8 27B · Q4_K_M (unsloth), reasoning mode · 50.9 tok/s · speculative · llama.cpp (MTP, 2 drafts) source
- Qwen3.8 27B · Q4_K_M (unsloth), no speculation · 42.9 tok/s · OpenJet megakernel source
- Qwen3.8 27B · UD-Q4_K_XL (GGUF) · 40.6 tok/s · speculative · llama.cpp, tuned source
- Qwen3.8 27B · Q4_K_M (unsloth), no speculation · 40.4 tok/s · llama.cpp source
- Qwen3.8 27B · Q4_K_M · 39 tok/s · llama.cpp server source
- Gemma 4 31B · Q4_K_M · 38.8 tok/s · llama.cpp b11658 (llama-bench) source
- Qwen3.8 27B · UD-Q4_K_XL (GGUF) · 38.8 tok/s · llama.cpp llama-bench, 1 GPU at 350 W source
- Qwen3.8 27B · Q4_K_M · 38.6 tok/s · llama.cpp build 11803 (a37cc4610) source
- Qwen3.8 27B · Unsloth UD-Q4_K_M, f16 KV · 38.3 tok/s · llama.cpp b11382 (Windows, CUDA 13), MTP off, -np 4 source
- Qwen3.8 27B · llama-bench tg128 (quant per result) · 38.2–71.6 tok/s · llama.cpp source
- Qwen3.8 27B · Q4_K_M · 38.1 tok/s · llama.cpp build 11803 (a37cc4610) source
- Qwen3.8 27B · AutoRound W4A16 · 32.6 tok/s · vLLM (stock) source
- gpt-oss 120B · Q4_K_M (58.5 GiB, larger than VRAM + RAM) · 18.1 tok/s · llama.cpp CUDA b11804, rest mmapped from NVMe page cache source
- DeepSeek R1 671B · Q2_K · 8.97 tok/s · llama.cpp CUDA source