Skip to content
RigCheck

Models / gpt-oss

Can an RTX 3060 12GB run gpt-oss 20B?

OpenAI's smaller open-weight MoE. The published MXFP4 checkpoint is 12.8 GiB.

Stored parameters
20.9B
Active parameters
3.6B
Q4 total (estimate)
13.4 GB
License
Apache 2.0

Model card. OpenAI's model card: 20.9B total parameters, 3.6B active, MXFP4 checkpoint 12.8 GiB. MoE weights are 4.25 bits.

Compatibility

HardwareQ4Q8FP16MXFP4
RTX 5090
NVIDIA · 32 GB VRAM
13.4 GB
Fits
estimate
Measured · 324 tok/s median
23.0 GB
Fits
estimate
No measured run
42.6 GB
Over
estimate
No measured run
13.6 GB
Fits
checkpoint
Measured · 283 tok/s median
RTX 3090
NVIDIA · 24 GB VRAM
13.4 GB
Fits
estimate
Measured · 207 tok/s median
23.0 GB
Tight
estimate
No measured run
42.6 GB
Over
estimate
No measured run
M3 Ultra 512GB
Apple · 512 GB unified
13.4 GB
Fits
estimate
No measured run
23.0 GB
Fits
estimate
No measured run
42.6 GB
Fits
estimate
No measured run
13.6 GB
Fits
checkpoint
Measured · 116 tok/s median
2× NVIDIA GB10
NVIDIA · 256 GB unified
13.4 GB
Fits
estimate
No measured run
23.0 GB
Fits
estimate
No measured run
42.6 GB
Fits
estimate
No measured run
13.6 GB
Fits
checkpoint
No measured run
M3 Ultra 256GB
Apple · 256 GB unified
13.4 GB
Fits
estimate
No measured run
23.0 GB
Fits
estimate
No measured run
42.6 GB
Fits
estimate
No measured run
13.6 GB
Fits
checkpoint
No measured run
M5 Ultra 256GB
Apple · 256 GB unified
13.4 GB
Fits
estimate
No measured run
23.0 GB
Fits
estimate
No measured run
42.6 GB
Fits
estimate
No measured run
13.6 GB
Fits
checkpoint
No measured run
M2 Ultra 192GB
Apple · 192 GB unified
13.4 GB
Fits
estimate
No measured run
23.0 GB
Fits
estimate
No measured run
42.6 GB
Fits
estimate
No measured run
13.6 GB
Fits
checkpoint
Measured · 116 tok/s median
NVIDIA GB10
NVIDIA · 128 GB unified
13.4 GB
Fits
estimate
No measured run
23.0 GB
Fits
estimate
No measured run
42.6 GB
Fits
estimate
No measured run
13.6 GB
Fits
checkpoint
No measured run
Ryzen AI Max+ 395 128GB
AMD · 128 GB unified
13.4 GB
Fits
estimate
No measured run
23.0 GB
Fits
estimate
No measured run
42.6 GB
Fits
estimate
No measured run
13.6 GB
Fits
checkpoint
No measured run
M1 Ultra 128GB
Apple · 128 GB unified
13.4 GB
Fits
estimate
No measured run
23.0 GB
Fits
estimate
No measured run
42.6 GB
Fits
estimate
No measured run
13.6 GB
Fits
checkpoint
No measured run
M5 Max 128GB
Apple · 128 GB unified
13.4 GB
Fits
estimate
No measured run
23.0 GB
Fits
estimate
No measured run
42.6 GB
Fits
estimate
No measured run
13.6 GB
Fits
checkpoint
No measured run
4× RTX 3090
NVIDIA · 96 GB VRAM
13.4 GB
Fits
estimate
No measured run
23.0 GB
Fits
estimate
No measured run
42.6 GB
Fits
estimate
No measured run
13.6 GB
Fits
checkpoint
No measured run
RTX Pro 6000 Blackwell
NVIDIA · 96 GB VRAM
13.4 GB
Fits
estimate
No measured run
23.0 GB
Fits
estimate
No measured run
42.6 GB
Fits
estimate
No measured run
13.6 GB
Fits
checkpoint
Measured · 287 tok/s median
M5 Ultra 96GB
Apple · 96 GB unified
13.4 GB
Fits
estimate
No measured run
23.0 GB
Fits
estimate
No measured run
42.6 GB
Fits
estimate
No measured run
13.6 GB
Fits
checkpoint
No measured run
Ryzen AI Max+ 395 64GB
AMD · 64 GB unified
13.4 GB
Fits
estimate
No measured run
23.0 GB
Fits
estimate
No measured run
42.6 GB
Fits
estimate
No measured run
13.6 GB
Fits
checkpoint
Measured · 67.2 tok/s median
2× RTX 3090
NVIDIA · 48 GB VRAM
13.4 GB
Fits
estimate
No measured run
23.0 GB
Fits
estimate
No measured run
42.6 GB
Fits
estimate
No measured run
13.6 GB
Fits
checkpoint
No measured run
M4 Pro 48GB
Apple · 48 GB unified
13.4 GB
Fits
estimate
No measured run
23.0 GB
Fits
estimate
No measured run
42.6 GB
Tight
estimate
No measured run
13.6 GB
Fits
checkpoint
No measured run
M4 Max 36GB
Apple · 36 GB unified
13.4 GB
Fits
estimate
No measured run
23.0 GB
Fits
estimate
No measured run
42.6 GB
Over
estimate
No measured run
13.6 GB
Fits
checkpoint
Measured · 92.4 tok/s median
M5 Max 36GB
Apple · 36 GB unified
13.4 GB
Fits
estimate
No measured run
23.0 GB
Fits
estimate
No measured run
42.6 GB
Over
estimate
No measured run
13.6 GB
Fits
checkpoint
No measured run
Radeon AI Pro R9700
AMD · 32 GB VRAM
13.4 GB
Fits
estimate
No measured run
23.0 GB
Fits
estimate
No measured run
42.6 GB
Over
estimate
No measured run
13.6 GB
Fits
checkpoint
No measured run
Intel Arc Pro B70
Intel · 32 GB VRAM
13.4 GB
Fits
estimate
No measured run
23.0 GB
Fits
estimate
No measured run
42.6 GB
Over
estimate
No measured run
13.6 GB
Fits
checkpoint
Measured · 134 tok/s median
M4 32GB
Apple · 32 GB unified
13.4 GB
Fits
estimate
No measured run
23.0 GB
Tight
estimate
No measured run
42.6 GB
Over
estimate
No measured run
13.6 GB
Fits
checkpoint
No measured run
M6 32GB
Apple · 32 GB unified
13.4 GB
Fits
estimate
No measured run
23.0 GB
Tight
estimate
No measured run
42.6 GB
Over
estimate
No measured run
13.6 GB
Fits
checkpoint
No measured run
RTX 4090
NVIDIA · 24 GB VRAM
13.4 GB
Fits
estimate
No measured run
23.0 GB
Tight
estimate
No measured run
42.6 GB
Over
estimate
No measured run
13.6 GB
Fits
checkpoint
Measured · 239 tok/s median
TITAN RTX
NVIDIA · 24 GB VRAM
13.4 GB
Fits
estimate
No measured run
23.0 GB
Tight
estimate
No measured run
42.6 GB
Over
estimate
No measured run
13.6 GB
Fits
checkpoint
No measured run
Radeon RX 7900 XTX
AMD · 24 GB VRAM
13.4 GB
Fits
estimate
No measured run
23.0 GB
Tight
estimate
No measured run
42.6 GB
Over
estimate
No measured run
13.6 GB
Fits
checkpoint
No measured run
M4 Pro 24GB
Apple · 24 GB unified
13.4 GB
Fits
estimate
No measured run
23.0 GB
Tight
estimate
No measured run
42.6 GB
Over
estimate
No measured run
13.6 GB
Fits
checkpoint
No measured run
RTX 4060 Ti 16GB
NVIDIA · 16 GB VRAM
13.4 GB
Fits
estimate
No measured run
23.0 GB
Over
estimate
No measured run
42.6 GB
Over
estimate
No measured run
13.6 GB
Fits
checkpoint
No measured run
RTX 5060 Ti 16GB
NVIDIA · 16 GB VRAM
13.4 GB
Fits
estimate
No measured run
23.0 GB
Over
estimate
No measured run
42.6 GB
Over
estimate
No measured run
13.6 GB
Fits
checkpoint
Measured · 112 tok/s median
RTX 5070 Ti
NVIDIA · 16 GB VRAM
13.4 GB
Fits
estimate
No measured run
23.0 GB
Over
estimate
No measured run
42.6 GB
Over
estimate
No measured run
M3 Pro 18GB
Apple · 18 GB unified
13.4 GB
Tight
estimate
No measured run
23.0 GB
Over
estimate
No measured run
42.6 GB
Over
estimate
No measured run
13.6 GB
Tight
checkpoint
No measured run
M4 16GB
Apple · 16 GB unified
13.4 GB
Tight
estimate
No measured run
23.0 GB
Over
estimate
No measured run
42.6 GB
Over
estimate
No measured run
13.6 GB
Tight
checkpoint
No measured run
RTX 3060 12GB
NVIDIA · 12 GB VRAM
13.4 GB
Over
estimate
No measured run
23.0 GB
Over
estimate
No measured run
42.6 GB
Over
estimate
No measured run
13.6 GB
Over
checkpoint
No measured run
RTX 3080 Ti
NVIDIA · 12 GB VRAM
13.4 GB
Over
estimate
No measured run
23.0 GB
Over
estimate
No measured run
42.6 GB
Over
estimate
No measured run
13.6 GB
Over
checkpoint
No measured run
RTX 4070 Ti
NVIDIA · 12 GB VRAM
13.4 GB
Over
estimate
No measured run
23.0 GB
Over
estimate
No measured run
42.6 GB
Over
estimate
No measured run
13.6 GB
Over
checkpoint
No measured run
RTX 2060 Super
NVIDIA · 8 GB VRAM
13.4 GB
Over
estimate
No measured run
23.0 GB
Over
estimate
No measured run
42.6 GB
Over
estimate
No measured run
13.6 GB
Over
checkpoint
No measured run

Measured runs

HardwareQuanttok/sRuntimeSource
RTX 5090reported as Q4_K_M324.2llama.cpp 0.7.0-devSource · checked · prefill 12638
RTX 5090llama-bench tg128 (quant per result)313llama.cppSource · checked
RTX Pro 6000 BlackwellMXFP4286.91llama.cppSource · checked · prefill 10673
RTX 5090MXFP4282.51llama.cppSource · checked · prefill 8834
RTX 4090not stated279.65llama.cppSource · checked · prefill 5908
RTX 4090MXFP4 (12.1 GB GGUF), reasoning low238.6llama.cpp b11669 (CUDA)Source · checked · prefill 9940
RTX 5070 TiMXFP4236.7llama.cpp b12128 (CUDA)Source · checked · prefill 8442
RTX 5070 TiMXFP4 (12.1 GB GGUF), reasoning low213.2llama.cpp b11669 (CUDA)Source · checked · prefill 7884
RTX 3090Q4_K_M (bartowski GGUF, MXFP4 experts)211.1llama.cpp CUDA b11804Source · checked · prefill 4232
RTX 3090reported as Q4_K_M207.2llama.cpp 0.7.0-devSource · checked · prefill 4606
RTX 4090llama-bench tg128 (quant per result)203llama.cppSource · checked
RTX 5070 TiMXFP4189.45llama.cppSource · checked · prefill 5914
Radeon AI Pro R9700not stated163.26llama.cppSource · checked · prefill 2613
RTX 4070 Tinot stated163.14llama.cppSource · checked · prefill 4421
RTX 3090MXFP4161.77llama.cppSource · checked · prefill 4772
Intel Arc Pro B70MXFP4134.2llama.cpp b12128 (SYCL)Source · checked · prefill 1296
M2 Ultra 192GBMXFP4116.08llama.cppSource · checked · prefill 1890
M3 Ultra 512GBMXFP4115.52llama.cpp (Metal, llama-bench)Source · checked · prefill 2308
M3 Ultra 512GBnot stated115.29llama.cppSource · checked · prefill 2238
RTX 5060 Ti 16GBMXFP4111.51llama.cppSource · checked · prefill 3696
M1 Ultra 128GBnot stated100.43llama.cppSource · checked · prefill 1595
NVIDIA GB10not stated96.2vLLM 0.31.0, 128K ctx, 1 userSource · checked
M4 Max 36GBMXFP492.36llama.cppSource · checked · prefill 1030
RTX 3060 12GBnot stated83.96llama.cppSource · checked · prefill 1589
Ryzen AI Max+ 395 64GBMXFP4 (12.1 GB GGUF), reasoning low67.2llama.cpp b11669 (Vulkan)Source · checked · prefill 1220