Models / gpt-oss
Can an RTX 3060 12GB run gpt-oss 20B?
OpenAI's smaller open-weight MoE. The published MXFP4 checkpoint is 12.8 GiB.
- Stored parameters
- 20.9B
- Active parameters
- 3.6B
- Q4 total (estimate)
- 13.4 GB
- License
- Apache 2.0
Model card. OpenAI's model card: 20.9B total parameters, 3.6B active, MXFP4 checkpoint 12.8 GiB. MoE weights are 4.25 bits.
Compatibility
| Hardware | Q4 | Q8 | FP16 | MXFP4 |
|---|---|---|---|---|
| RTX 5090 NVIDIA · 32 GB VRAM | 23.0 GB Fits estimate No measured run | 42.6 GB Over estimate No measured run | ||
| RTX 3090 NVIDIA · 24 GB VRAM | 23.0 GB Tight estimate No measured run | 42.6 GB Over estimate No measured run | ||
| M3 Ultra 512GB Apple · 512 GB unified | 13.4 GB Fits estimate No measured run | 23.0 GB Fits estimate No measured run | 42.6 GB Fits estimate No measured run | |
| 2× NVIDIA GB10 NVIDIA · 256 GB unified | 13.4 GB Fits estimate No measured run | 23.0 GB Fits estimate No measured run | 42.6 GB Fits estimate No measured run | 13.6 GB Fits checkpoint No measured run |
| M3 Ultra 256GB Apple · 256 GB unified | 13.4 GB Fits estimate No measured run | 23.0 GB Fits estimate No measured run | 42.6 GB Fits estimate No measured run | 13.6 GB Fits checkpoint No measured run |
| M5 Ultra 256GB Apple · 256 GB unified | 13.4 GB Fits estimate No measured run | 23.0 GB Fits estimate No measured run | 42.6 GB Fits estimate No measured run | 13.6 GB Fits checkpoint No measured run |
| M2 Ultra 192GB Apple · 192 GB unified | 13.4 GB Fits estimate No measured run | 23.0 GB Fits estimate No measured run | 42.6 GB Fits estimate No measured run | |
| NVIDIA GB10 NVIDIA · 128 GB unified | 13.4 GB Fits estimate No measured run | 23.0 GB Fits estimate No measured run | 42.6 GB Fits estimate No measured run | 13.6 GB Fits checkpoint No measured run |
| Ryzen AI Max+ 395 128GB AMD · 128 GB unified | 13.4 GB Fits estimate No measured run | 23.0 GB Fits estimate No measured run | 42.6 GB Fits estimate No measured run | 13.6 GB Fits checkpoint No measured run |
| M1 Ultra 128GB Apple · 128 GB unified | 13.4 GB Fits estimate No measured run | 23.0 GB Fits estimate No measured run | 42.6 GB Fits estimate No measured run | 13.6 GB Fits checkpoint No measured run |
| M5 Max 128GB Apple · 128 GB unified | 13.4 GB Fits estimate No measured run | 23.0 GB Fits estimate No measured run | 42.6 GB Fits estimate No measured run | 13.6 GB Fits checkpoint No measured run |
| 4× RTX 3090 NVIDIA · 96 GB VRAM | 13.4 GB Fits estimate No measured run | 23.0 GB Fits estimate No measured run | 42.6 GB Fits estimate No measured run | 13.6 GB Fits checkpoint No measured run |
| RTX Pro 6000 Blackwell NVIDIA · 96 GB VRAM | 13.4 GB Fits estimate No measured run | 23.0 GB Fits estimate No measured run | 42.6 GB Fits estimate No measured run | |
| M5 Ultra 96GB Apple · 96 GB unified | 13.4 GB Fits estimate No measured run | 23.0 GB Fits estimate No measured run | 42.6 GB Fits estimate No measured run | 13.6 GB Fits checkpoint No measured run |
| Ryzen AI Max+ 395 64GB AMD · 64 GB unified | 13.4 GB Fits estimate No measured run | 23.0 GB Fits estimate No measured run | 42.6 GB Fits estimate No measured run | |
| 2× RTX 3090 NVIDIA · 48 GB VRAM | 13.4 GB Fits estimate No measured run | 23.0 GB Fits estimate No measured run | 42.6 GB Fits estimate No measured run | 13.6 GB Fits checkpoint No measured run |
| M4 Pro 48GB Apple · 48 GB unified | 13.4 GB Fits estimate No measured run | 23.0 GB Fits estimate No measured run | 42.6 GB Tight estimate No measured run | 13.6 GB Fits checkpoint No measured run |
| M4 Max 36GB Apple · 36 GB unified | 13.4 GB Fits estimate No measured run | 23.0 GB Fits estimate No measured run | 42.6 GB Over estimate No measured run | |
| M5 Max 36GB Apple · 36 GB unified | 13.4 GB Fits estimate No measured run | 23.0 GB Fits estimate No measured run | 42.6 GB Over estimate No measured run | 13.6 GB Fits checkpoint No measured run |
| Radeon AI Pro R9700 AMD · 32 GB VRAM | 13.4 GB Fits estimate No measured run | 23.0 GB Fits estimate No measured run | 42.6 GB Over estimate No measured run | 13.6 GB Fits checkpoint No measured run |
| Intel Arc Pro B70 Intel · 32 GB VRAM | 13.4 GB Fits estimate No measured run | 23.0 GB Fits estimate No measured run | 42.6 GB Over estimate No measured run | |
| M4 32GB Apple · 32 GB unified | 13.4 GB Fits estimate No measured run | 23.0 GB Tight estimate No measured run | 42.6 GB Over estimate No measured run | 13.6 GB Fits checkpoint No measured run |
| M6 32GB Apple · 32 GB unified | 13.4 GB Fits estimate No measured run | 23.0 GB Tight estimate No measured run | 42.6 GB Over estimate No measured run | 13.6 GB Fits checkpoint No measured run |
| RTX 4090 NVIDIA · 24 GB VRAM | 13.4 GB Fits estimate No measured run | 23.0 GB Tight estimate No measured run | 42.6 GB Over estimate No measured run | |
| TITAN RTX NVIDIA · 24 GB VRAM | 13.4 GB Fits estimate No measured run | 23.0 GB Tight estimate No measured run | 42.6 GB Over estimate No measured run | 13.6 GB Fits checkpoint No measured run |
| Radeon RX 7900 XTX AMD · 24 GB VRAM | 13.4 GB Fits estimate No measured run | 23.0 GB Tight estimate No measured run | 42.6 GB Over estimate No measured run | 13.6 GB Fits checkpoint No measured run |
| M4 Pro 24GB Apple · 24 GB unified | 13.4 GB Fits estimate No measured run | 23.0 GB Tight estimate No measured run | 42.6 GB Over estimate No measured run | 13.6 GB Fits checkpoint No measured run |
| RTX 4060 Ti 16GB NVIDIA · 16 GB VRAM | 13.4 GB Fits estimate No measured run | 23.0 GB Over estimate No measured run | 42.6 GB Over estimate No measured run | 13.6 GB Fits checkpoint No measured run |
| RTX 5060 Ti 16GB NVIDIA · 16 GB VRAM | 13.4 GB Fits estimate No measured run | 23.0 GB Over estimate No measured run | 42.6 GB Over estimate No measured run | |
| RTX 5070 Ti NVIDIA · 16 GB VRAM | 13.4 GB Fits estimate No measured run | 23.0 GB Over estimate No measured run | 42.6 GB Over estimate No measured run | |
| M3 Pro 18GB Apple · 18 GB unified | 13.4 GB Tight estimate No measured run | 23.0 GB Over estimate No measured run | 42.6 GB Over estimate No measured run | 13.6 GB Tight checkpoint No measured run |
| M4 16GB Apple · 16 GB unified | 13.4 GB Tight estimate No measured run | 23.0 GB Over estimate No measured run | 42.6 GB Over estimate No measured run | 13.6 GB Tight checkpoint No measured run |
| RTX 3060 12GB NVIDIA · 12 GB VRAM | 13.4 GB Over estimate No measured run | 23.0 GB Over estimate No measured run | 42.6 GB Over estimate No measured run | 13.6 GB Over checkpoint No measured run |
| RTX 3080 Ti NVIDIA · 12 GB VRAM | 13.4 GB Over estimate No measured run | 23.0 GB Over estimate No measured run | 42.6 GB Over estimate No measured run | 13.6 GB Over checkpoint No measured run |
| RTX 4070 Ti NVIDIA · 12 GB VRAM | 13.4 GB Over estimate No measured run | 23.0 GB Over estimate No measured run | 42.6 GB Over estimate No measured run | 13.6 GB Over checkpoint No measured run |
| RTX 2060 Super NVIDIA · 8 GB VRAM | 13.4 GB Over estimate No measured run | 23.0 GB Over estimate No measured run | 42.6 GB Over estimate No measured run | 13.6 GB Over checkpoint No measured run |
Measured runs
| Hardware | Quant | tok/s | Runtime | Source |
|---|---|---|---|---|
| RTX 5090 | reported as Q4_K_M | 324.2 | llama.cpp 0.7.0-dev | Source · checked · prefill 12638 |
| RTX 5090 | llama-bench tg128 (quant per result) | 313 | llama.cpp | Source · checked |
| RTX Pro 6000 Blackwell | MXFP4 | 286.91 | llama.cpp | Source · checked · prefill 10673 |
| RTX 5090 | MXFP4 | 282.51 | llama.cpp | Source · checked · prefill 8834 |
| RTX 4090 | not stated | 279.65 | llama.cpp | Source · checked · prefill 5908 |
| RTX 4090 | MXFP4 (12.1 GB GGUF), reasoning low | 238.6 | llama.cpp b11669 (CUDA) | Source · checked · prefill 9940 |
| RTX 5070 Ti | MXFP4 | 236.7 | llama.cpp b12128 (CUDA) | Source · checked · prefill 8442 |
| RTX 5070 Ti | MXFP4 (12.1 GB GGUF), reasoning low | 213.2 | llama.cpp b11669 (CUDA) | Source · checked · prefill 7884 |
| RTX 3090 | Q4_K_M (bartowski GGUF, MXFP4 experts) | 211.1 | llama.cpp CUDA b11804 | Source · checked · prefill 4232 |
| RTX 3090 | reported as Q4_K_M | 207.2 | llama.cpp 0.7.0-dev | Source · checked · prefill 4606 |
| RTX 4090 | llama-bench tg128 (quant per result) | 203 | llama.cpp | Source · checked |
| RTX 5070 Ti | MXFP4 | 189.45 | llama.cpp | Source · checked · prefill 5914 |
| Radeon AI Pro R9700 | not stated | 163.26 | llama.cpp | Source · checked · prefill 2613 |
| RTX 4070 Ti | not stated | 163.14 | llama.cpp | Source · checked · prefill 4421 |
| RTX 3090 | MXFP4 | 161.77 | llama.cpp | Source · checked · prefill 4772 |
| Intel Arc Pro B70 | MXFP4 | 134.2 | llama.cpp b12128 (SYCL) | Source · checked · prefill 1296 |
| M2 Ultra 192GB | MXFP4 | 116.08 | llama.cpp | Source · checked · prefill 1890 |
| M3 Ultra 512GB | MXFP4 | 115.52 | llama.cpp (Metal, llama-bench) | Source · checked · prefill 2308 |
| M3 Ultra 512GB | not stated | 115.29 | llama.cpp | Source · checked · prefill 2238 |
| RTX 5060 Ti 16GB | MXFP4 | 111.51 | llama.cpp | Source · checked · prefill 3696 |
| M1 Ultra 128GB | not stated | 100.43 | llama.cpp | Source · checked · prefill 1595 |
| NVIDIA GB10 | not stated | 96.2 | vLLM 0.31.0, 128K ctx, 1 user | Source · checked |
| M4 Max 36GB | MXFP4 | 92.36 | llama.cpp | Source · checked · prefill 1030 |
| RTX 3060 12GB | not stated | 83.96 | llama.cpp | Source · checked · prefill 1589 |
| Ryzen AI Max+ 395 64GB | MXFP4 (12.1 GB GGUF), reasoning low | 67.2 | llama.cpp b11669 (Vulkan) | Source · checked · prefill 1220 |