Hardware / NVIDIA
Can 2× RTX 3090 run gpt-oss 120B?
2× RTX 3090 has 48 GB VRAM. About 43.2 GB is treated as usable. Two 24 GB cards, 48 GB combined. Tensor-parallel runs in the dataset are kept on this entry. The title asks about gpt-oss 120B because it is the largest catalog model still in range of this memory.
RigBase's row for this class cites a builder who described two used 3090s as about 2000 EUR all-in. That is one report, not a current street price. Source
Manufacturer page for 2× RTX 3090
Compatibility
| Model | Q4 | Q8 | FP16 | MXFP4 |
|---|---|---|---|---|
| Llama 3 70B Llama · 70B stored | 77.6 GB Over estimate · KV at 8k No measured run | 143 GB Over estimate · KV at 8k No measured run | — | |
| Llama 3.3 70B Llama · 70B stored | 77.6 GB Over estimate · KV at 8k No measured run | 143 GB Over estimate · KV at 8k No measured run | — | |
| Qwen3.8 27B Qwen · 27B stored
| 29.9 GB Fits estimate · KV at 8k No measured run | 55.3 GB Over estimate · KV at 8k No measured run | — | |
| Qwen3.6 35B-A3B Qwen · 35B stored · 3B active | 22.1 GB Fits estimate · KV at 8k No measured run | 38.1 GB Fits estimate · KV at 8k No measured run | 70.9 GB Over estimate · KV at 8k No measured run | — |
| Gemma 4 31B Gemma · 30.7B stored | 19.4 GB Fits estimate No measured run | 33.4 GB Fits estimate No measured run | 62.1 GB Over estimate No measured run | — |
| Qwen3 30B-A3B Qwen · 30B stored · 3B active | 18.9 GB Fits estimate No measured run | 32.6 GB Fits estimate No measured run | 60.8 GB Over estimate No measured run | — |
| Gemma 4 26B-A4B Gemma · 25.2B stored · 3.8B active | 16.0 GB Fits estimate No measured run | 27.5 GB Fits estimate No measured run | 51.1 GB Over estimate No measured run | — |
| gpt-oss 20B gpt-oss · 20.9B stored · 3.6B active | 13.4 GB Fits estimate No measured run | 23.0 GB Fits estimate No measured run | 42.6 GB Fits estimate No measured run | 13.6 GB Fits checkpoint No measured run |
| Qwen3 14B Qwen · 14B stored | 9.2 GB Fits estimate No measured run | 15.6 GB Fits estimate No measured run | 28.8 GB Fits estimate No measured run | — |
| DeepSeek R1 Distill 14B DeepSeek · 14B stored | 9.2 GB Fits estimate No measured run | 15.6 GB Fits estimate No measured run | 28.8 GB Fits estimate No measured run | — |
| Gemma 3 12B Gemma · 12B stored | 8.0 GB Fits estimate No measured run | 13.5 GB Fits estimate No measured run | 24.8 GB Fits estimate No measured run | — |
| Gemma 4 12B Gemma · 12B stored | 8.0 GB Fits estimate No measured run | 13.4 GB Fits estimate No measured run | 24.6 GB Fits estimate No measured run | — |
| Llama 3 8B Llama · 8B stored | 6.6 GB Fits estimate · KV at 8k No measured run | 10.3 GB Fits estimate · KV at 8k No measured run | 17.8 GB Fits estimate · KV at 8k No measured run | — |
| Llama 3.1 8B Llama · 8B stored | 6.6 GB Fits estimate · KV at 8k No measured run | 10.3 GB Fits estimate · KV at 8k No measured run | 17.8 GB Fits estimate · KV at 8k No measured run | — |
| Qwen3 8B Qwen · 8B stored | 5.6 GB Fits estimate No measured run | 9.3 GB Fits estimate No measured run | 16.8 GB Fits estimate No measured run | — |
| Gemma 4 E4B Gemma · 8B stored | 5.6 GB Fits estimate No measured run | 9.3 GB Fits estimate No measured run | 16.8 GB Fits estimate No measured run | — |
| DeepSeek R1 Distill 8B DeepSeek · 8B stored | 6.6 GB Fits estimate · KV at 8k No measured run | 10.3 GB Fits estimate · KV at 8k No measured run | 17.8 GB Fits estimate · KV at 8k No measured run | — |
| Llama 2 7B Llama · 7B stored | 9.0 GB Fits estimate · KV at 8k No measured run | 12.2 GB Fits estimate · KV at 8k No measured run | 18.8 GB Fits estimate · KV at 8k No measured run | — |
| Llama 3.2 3B Llama · 3B stored | 3.4 GB Fits estimate · KV at 8k No measured run | 4.8 GB Fits estimate · KV at 8k No measured run | 7.6 GB Fits estimate · KV at 8k No measured run | — |
| Llama 3.1 70B Llama · 70B stored | 45.7 GB Tight estimate · KV at 8k No measured run | 77.6 GB Over estimate · KV at 8k No measured run | 143 GB Over estimate · KV at 8k No measured run | — |
| DeepSeek R1 Distill 70B DeepSeek · 70B stored | 45.7 GB Tight estimate · KV at 8k No measured run | 77.6 GB Over estimate · KV at 8k No measured run | 143 GB Over estimate · KV at 8k No measured run | — |
| DeepSeek R1 671B DeepSeek · 671B stored · 37B active | 408 GB Over estimate No measured run | 714 GB Over estimate No measured run | 1343 GB Over estimate No measured run | — |
| Llama 3.1 405B Llama · 405B stored | 250 GB Over estimate · KV at 8k No measured run | 435 GB Over estimate · KV at 8k No measured run | 815 GB Over estimate · KV at 8k No measured run | — |
| Qwen3 235B-A22B Qwen · 235B stored · 22B active | 143 GB Over estimate No measured run | 250 GB Over estimate No measured run | 471 GB Over estimate No measured run | — |
| gpt-oss 120B gpt-oss · 117B stored · 5.1B active | 71.6 GB Over estimate No measured run | 125 GB Over estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Over checkpoint No measured run |
Measured runs on this device
- Qwen3.8 27B · INT4 (RedHat INT4 checkpoint, QEngine layout), fp8 KV · 127.5 tok/s · speculative · QEngine 1.3.2 (own collectives, P2P over NVLink), MTP head source
- Qwen3.8 27B · W4A16 AutoRound + DFlash2 · 91.2 tok/s · speculative · vLLM TP2, 1 stream source
- Qwen3.8 27B · AWQ + DSpark · 69.9 tok/s · speculative · SGLang TP2, 1 stream source
- Qwen3.8 27B · Q6_K_L (bartowski GGUF) · 55.4 tok/s · speculative · llama.cpp b10666, -sm tensor (NCCL) source
- Qwen3.8 27B · Q6_K_L (bartowski GGUF) · 48.2 tok/s · llama.cpp b10666, -sm tensor (NCCL) source
- Qwen3.8 27B · UD-Q4_K_M (GGUF) · 40.556 tok/s · llama.cpp b10775, 2 GPU -sm layer source
- Qwen3.8 27B · UD-Q4_K_M (GGUF) · 39.157 tok/s · llama.cpp b10775, 1 GPU source
- Qwen3.8 27B · Q6_K_L (bartowski GGUF) · 32.4 tok/s · llama.cpp b10666, -sm layer, out of the box source
- Qwen3.8 27B · UD-Q4_K_M (GGUF) · 31.252 tok/s · llama.cpp b10775, 2 GPU -sm tensor + NCCL 2.27.5 source
- Llama 3 70B · AWQ 4-bit · 21.2 tok/s · vLLM (TP=2) source
- Llama 3.3 70B · not stated · 20 tok/s · ExLlamaV2 (TP) source
- Llama 3.3 70B · Q4_K_M · 19.1 tok/s · llama.cpp 0.7.0-dev source
- Llama 3 70B · Q4_K_M · 16.95 tok/s · Ollama source