Models / gpt-oss
Can 2× RTX 3090 run gpt-oss 120B?
OpenAI's larger open-weight MoE. Native MXFP4 checkpoint is 60.8 GiB, not a 24 GB card.
- Stored parameters
- 117B
- Active parameters
- 5.1B
- Q4 total (estimate)
- 71.6 GB
- License
- Apache 2.0
Model card. OpenAI's model card: 116.8B total, 5.1B active, MXFP4 checkpoint 60.8 GiB.
Compatibility
| Hardware | Q4 | Q8 | FP16 | MXFP4 |
|---|---|---|---|---|
| RTX 3090 NVIDIA · 24 GB VRAM | 125 GB Over estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Over checkpoint No measured run | |
| M3 Ultra 512GB Apple · 512 GB unified | 71.6 GB Fits estimate No measured run | 125 GB Fits estimate No measured run | 234 GB Fits estimate No measured run | 61.5 GB Fits checkpoint No measured run |
| 2× NVIDIA GB10 NVIDIA · 256 GB unified | 71.6 GB Fits estimate No measured run | 125 GB Fits estimate No measured run | 234 GB Tight estimate No measured run | 61.5 GB Fits checkpoint No measured run |
| M3 Ultra 256GB Apple · 256 GB unified | 71.6 GB Fits estimate No measured run | 125 GB Fits estimate No measured run | 234 GB Tight estimate No measured run | 61.5 GB Fits checkpoint No measured run |
| M5 Ultra 256GB Apple · 256 GB unified | 71.6 GB Fits estimate No measured run | 125 GB Fits estimate No measured run | 234 GB Tight estimate No measured run | |
| M2 Ultra 192GB Apple · 192 GB unified | 71.6 GB Fits estimate No measured run | 125 GB Fits estimate No measured run | 234 GB Over estimate No measured run | |
| NVIDIA GB10 NVIDIA · 128 GB unified | 71.6 GB Fits estimate No measured run | 125 GB Tight estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Fits checkpoint No measured run |
| Ryzen AI Max+ 395 128GB AMD · 128 GB unified | 71.6 GB Fits estimate No measured run | 125 GB Tight estimate No measured run | 234 GB Over estimate No measured run | |
| M1 Ultra 128GB Apple · 128 GB unified | 71.6 GB Fits estimate No measured run | 125 GB Tight estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Fits checkpoint No measured run |
| M5 Max 128GB Apple · 128 GB unified | 71.6 GB Fits estimate No measured run | 125 GB Tight estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Fits checkpoint No measured run |
| 4× RTX 3090 NVIDIA · 96 GB VRAM | 71.6 GB Fits estimate No measured run | 125 GB Over estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Fits checkpoint No measured run |
| RTX Pro 6000 Blackwell NVIDIA · 96 GB VRAM | 71.6 GB Fits estimate No measured run | 125 GB Over estimate No measured run | 234 GB Over estimate No measured run | |
| M5 Ultra 96GB Apple · 96 GB unified | 71.6 GB Tight estimate No measured run | 125 GB Over estimate No measured run | 234 GB Over estimate No measured run | |
| Ryzen AI Max+ 395 64GB AMD · 64 GB unified | 71.6 GB Over estimate No measured run | 125 GB Over estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Tight checkpoint No measured run |
| 2× RTX 3090 NVIDIA · 48 GB VRAM | 71.6 GB Over estimate No measured run | 125 GB Over estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Over checkpoint No measured run |
| M4 Pro 48GB Apple · 48 GB unified | 71.6 GB Over estimate No measured run | 125 GB Over estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Over checkpoint No measured run |
| M4 Max 36GB Apple · 36 GB unified | 71.6 GB Over estimate No measured run | 125 GB Over estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Over checkpoint No measured run |
| M5 Max 36GB Apple · 36 GB unified | 71.6 GB Over estimate No measured run | 125 GB Over estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Over checkpoint No measured run |
| RTX 5090 NVIDIA · 32 GB VRAM | 71.6 GB Over estimate No measured run | 125 GB Over estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Over checkpoint No measured run |
| Radeon AI Pro R9700 AMD · 32 GB VRAM | 71.6 GB Over estimate No measured run | 125 GB Over estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Over checkpoint No measured run |
| Intel Arc Pro B70 Intel · 32 GB VRAM | 71.6 GB Over estimate No measured run | 125 GB Over estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Over checkpoint No measured run |
| M4 32GB Apple · 32 GB unified | 71.6 GB Over estimate No measured run | 125 GB Over estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Over checkpoint No measured run |
| M6 32GB Apple · 32 GB unified | 71.6 GB Over estimate No measured run | 125 GB Over estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Over checkpoint No measured run |
| RTX 4090 NVIDIA · 24 GB VRAM | 71.6 GB Over estimate No measured run | 125 GB Over estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Over checkpoint No measured run |
| TITAN RTX NVIDIA · 24 GB VRAM | 71.6 GB Over estimate No measured run | 125 GB Over estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Over checkpoint No measured run |
| Radeon RX 7900 XTX AMD · 24 GB VRAM | 71.6 GB Over estimate No measured run | 125 GB Over estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Over checkpoint No measured run |
| M4 Pro 24GB Apple · 24 GB unified | 71.6 GB Over estimate No measured run | 125 GB Over estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Over checkpoint No measured run |
| M3 Pro 18GB Apple · 18 GB unified | 71.6 GB Over estimate No measured run | 125 GB Over estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Over checkpoint No measured run |
| RTX 4060 Ti 16GB NVIDIA · 16 GB VRAM | 71.6 GB Over estimate No measured run | 125 GB Over estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Over checkpoint No measured run |
| RTX 5060 Ti 16GB NVIDIA · 16 GB VRAM | 71.6 GB Over estimate No measured run | 125 GB Over estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Over checkpoint No measured run |
| RTX 5070 Ti NVIDIA · 16 GB VRAM | 71.6 GB Over estimate No measured run | 125 GB Over estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Over checkpoint No measured run |
| M4 16GB Apple · 16 GB unified | 71.6 GB Over estimate No measured run | 125 GB Over estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Over checkpoint No measured run |
| RTX 3060 12GB NVIDIA · 12 GB VRAM | 71.6 GB Over estimate No measured run | 125 GB Over estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Over checkpoint No measured run |
| RTX 3080 Ti NVIDIA · 12 GB VRAM | 71.6 GB Over estimate No measured run | 125 GB Over estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Over checkpoint No measured run |
| RTX 4070 Ti NVIDIA · 12 GB VRAM | 71.6 GB Over estimate No measured run | 125 GB Over estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Over checkpoint No measured run |
| RTX 2060 Super NVIDIA · 8 GB VRAM | 71.6 GB Over estimate No measured run | 125 GB Over estimate No measured run | 234 GB Over estimate No measured run | 61.5 GB Over checkpoint No measured run |
Measured runs
| Hardware | Quant | tok/s | Runtime | Source |
|---|---|---|---|---|
| RTX Pro 6000 Blackwell | MXFP4 | 196.31 | llama.cpp | Source · checked · prefill 5316 |
| M5 Ultra 256GB | MXFP4-Q8 MLX (mlx-community) | 121.4 | mlx-lm 0.32.2 (MLX 0.32.4) | Source · checked · prefill 5143 |
| M5 Ultra 96GB | MXFP4 (Ollama gpt-oss:120b, 65 GB) | 95 | Ollama 0.40.13 (llamacpp runner) | Source · checked · prefill 2588 |
| M2 Ultra 192GB | MXFP4 | 79.68 | llama.cpp | Source · checked · prefill 1101 |
| Ryzen AI Max+ 395 128GB | MXFP4 | 44.7 | llama.cpp b11213 Vulkan | Source · checked |
| NVIDIA GB10 | not stated | 44.4 | vLLM 0.31.0, 128K ctx, 1 user | Source · checked |
| RTX 3090 | Q4_K_M (58.5 GiB, larger than VRAM + RAM) | 18.1 | llama.cpp CUDA b11804, rest mmapped from NVMe page cache | Source · checked · prefill 96.0 |