Models / Gemma
Can an RTX 3060 12GB run Gemma 4 26B-A4B?
Gemma 4 MoE. 25.2B stored, 3.8B active per token.
- Stored parameters
- 25.2B
- Active parameters
- 3.8B
- Q4 total (estimate)
- 16.0 GB
- License
- See model card
Model card. Google's card: 25.2B total parameters, 3.8B active, 256k context.
Compatibility
| Hardware | Q4 | Q8 | FP16 |
|---|---|---|---|
| Ryzen AI Max+ 395 128GB AMD · 128 GB unified | 27.5 GB Fits estimate No measured run | 51.1 GB Fits estimate No measured run | |
| Ryzen AI Max+ 395 64GB AMD · 64 GB unified | 27.5 GB Fits estimate No measured run | 51.1 GB Tight estimate No measured run | |
| RTX 5090 NVIDIA · 32 GB VRAM | 27.5 GB Fits estimate No measured run | 51.1 GB Over estimate No measured run | |
| RTX 4090 NVIDIA · 24 GB VRAM | 27.5 GB Over estimate No measured run | 51.1 GB Over estimate No measured run | |
| TITAN RTX NVIDIA · 24 GB VRAM | 27.5 GB Over estimate No measured run | 51.1 GB Over estimate No measured run | |
| RTX 5070 Ti NVIDIA · 16 GB VRAM | 27.5 GB Over estimate No measured run | 51.1 GB Over estimate No measured run | |
| M3 Ultra 512GB Apple · 512 GB unified | 16.0 GB Fits estimate No measured run | 27.5 GB Fits estimate No measured run | 51.1 GB Fits estimate No measured run |
| 2× NVIDIA GB10 NVIDIA · 256 GB unified | 16.0 GB Fits estimate No measured run | 27.5 GB Fits estimate No measured run | 51.1 GB Fits estimate No measured run |
| M3 Ultra 256GB Apple · 256 GB unified | 16.0 GB Fits estimate No measured run | 27.5 GB Fits estimate No measured run | 51.1 GB Fits estimate No measured run |
| M5 Ultra 256GB Apple · 256 GB unified | 16.0 GB Fits estimate No measured run | 27.5 GB Fits estimate No measured run | 51.1 GB Fits estimate No measured run |
| M2 Ultra 192GB Apple · 192 GB unified | 16.0 GB Fits estimate No measured run | 27.5 GB Fits estimate No measured run | 51.1 GB Fits estimate No measured run |
| NVIDIA GB10 NVIDIA · 128 GB unified | 16.0 GB Fits estimate No measured run | 27.5 GB Fits estimate No measured run | 51.1 GB Fits estimate No measured run |
| M1 Ultra 128GB Apple · 128 GB unified | 16.0 GB Fits estimate No measured run | 27.5 GB Fits estimate No measured run | 51.1 GB Fits estimate No measured run |
| M5 Max 128GB Apple · 128 GB unified | 16.0 GB Fits estimate No measured run | 27.5 GB Fits estimate No measured run | 51.1 GB Fits estimate No measured run |
| 4× RTX 3090 NVIDIA · 96 GB VRAM | 16.0 GB Fits estimate No measured run | 27.5 GB Fits estimate No measured run | 51.1 GB Fits estimate No measured run |
| RTX Pro 6000 Blackwell NVIDIA · 96 GB VRAM | 16.0 GB Fits estimate No measured run | 27.5 GB Fits estimate No measured run | 51.1 GB Fits estimate No measured run |
| M5 Ultra 96GB Apple · 96 GB unified | 16.0 GB Fits estimate No measured run | 27.5 GB Fits estimate No measured run | 51.1 GB Fits estimate No measured run |
| 2× RTX 3090 NVIDIA · 48 GB VRAM | 16.0 GB Fits estimate No measured run | 27.5 GB Fits estimate No measured run | 51.1 GB Over estimate No measured run |
| M4 Pro 48GB Apple · 48 GB unified | 16.0 GB Fits estimate No measured run | 27.5 GB Fits estimate No measured run | 51.1 GB Over estimate No measured run |
| M4 Max 36GB Apple · 36 GB unified | 16.0 GB Fits estimate No measured run | 27.5 GB Tight estimate No measured run | 51.1 GB Over estimate No measured run |
| M5 Max 36GB Apple · 36 GB unified | 16.0 GB Fits estimate No measured run | 27.5 GB Tight estimate No measured run | 51.1 GB Over estimate No measured run |
| Radeon AI Pro R9700 AMD · 32 GB VRAM | 16.0 GB Fits estimate No measured run | 27.5 GB Fits estimate No measured run | 51.1 GB Over estimate No measured run |
| Intel Arc Pro B70 Intel · 32 GB VRAM | 16.0 GB Fits estimate No measured run | 51.1 GB Over estimate No measured run | |
| M4 32GB Apple · 32 GB unified | 16.0 GB Fits estimate No measured run | 27.5 GB Tight estimate No measured run | 51.1 GB Over estimate No measured run |
| M6 32GB Apple · 32 GB unified | 16.0 GB Fits estimate No measured run | 27.5 GB Tight estimate No measured run | 51.1 GB Over estimate No measured run |
| RTX 3090 NVIDIA · 24 GB VRAM | 16.0 GB Fits estimate No measured run | 27.5 GB Over estimate No measured run | 51.1 GB Over estimate No measured run |
| Radeon RX 7900 XTX AMD · 24 GB VRAM | 16.0 GB Fits estimate No measured run | 27.5 GB Over estimate No measured run | 51.1 GB Over estimate No measured run |
| M4 Pro 24GB Apple · 24 GB unified | 16.0 GB Fits estimate No measured run | 27.5 GB Over estimate No measured run | 51.1 GB Over estimate No measured run |
| M3 Pro 18GB Apple · 18 GB unified | 16.0 GB Tight estimate No measured run | 27.5 GB Over estimate No measured run | 51.1 GB Over estimate No measured run |
| RTX 4060 Ti 16GB NVIDIA · 16 GB VRAM | 16.0 GB Over estimate No measured run | 27.5 GB Over estimate No measured run | 51.1 GB Over estimate No measured run |
| RTX 5060 Ti 16GB NVIDIA · 16 GB VRAM | 16.0 GB Over estimate No measured run | 27.5 GB Over estimate No measured run | 51.1 GB Over estimate No measured run |
| M4 16GB Apple · 16 GB unified | 16.0 GB Over estimate No measured run | 27.5 GB Over estimate No measured run | 51.1 GB Over estimate No measured run |
| RTX 3060 12GB NVIDIA · 12 GB VRAM | 16.0 GB Over estimate No measured run | 27.5 GB Over estimate No measured run | 51.1 GB Over estimate No measured run |
| RTX 3080 Ti NVIDIA · 12 GB VRAM | 16.0 GB Over estimate No measured run | 27.5 GB Over estimate No measured run | 51.1 GB Over estimate No measured run |
| RTX 4070 Ti NVIDIA · 12 GB VRAM | 16.0 GB Over estimate No measured run | 27.5 GB Over estimate No measured run | 51.1 GB Over estimate No measured run |
| RTX 2060 Super NVIDIA · 8 GB VRAM | 16.0 GB Over estimate No measured run | 27.5 GB Over estimate No measured run | 51.1 GB Over estimate No measured run |
Measured runs
| Hardware | Quant | tok/s | Runtime | Source |
|---|---|---|---|---|
| RTX 5090 | llama-bench tg128 (quant per result) | 235 | llama.cpp | Source · checked |
| RTX 5090 | Q4_K_M (text-only build), f16 KV | 232.4 | llama-server b10448 | Source · checked · prefill 9719 |
| M5 Max 128GB | 4-bit MLX (QAT) | 166 · speculative | Rapid-MLX 0.16.1 + DFlash | Source · checked |
| RTX 4090 | llama-bench tg128 (quant per result) | 164 | llama.cpp | Source · checked |
| RTX 4090 | UD-IQ4_XS (13.6 GB) | 161.4 | llama.cpp b11669 (CUDA) | Source · checked · prefill 7257 |
| Intel Arc Pro B70 | UD-Q8_K_XL | 122.16 | llama.cpp SYCL (pinned build, oneAPI 2026.0) | Source · checked · prefill 1168 |
| TITAN RTX | Unsloth UD-IQ4_XS | 96.6 | llama.cpp b11222 (CUDA) | Source · checked · prefill 2674 |
| RTX 5070 Ti | UD-IQ4_XS (13.6 GB) | 59.4 | llama.cpp b11669 (CUDA), partial CPU offload | Source · checked · prefill 2199 |
| NVIDIA GB10 | + official MTP drafter (k=3) | 56.5 · speculative | vLLM 0.31.0, 128K ctx, 1 user | Source · checked |
| Ryzen AI Max+ 395 128GB | 4-bit | 51.8 | Ollama 0.34.0 | Source · checked |
| M6 32GB | QAT (google/gemma-4-26b-a4b-qat) | 45.5 | LM Studio | Source · checked |
| Ryzen AI Max+ 395 64GB | UD-IQ4_XS (13.6 GB) | 43.3 | llama.cpp b11669 (Vulkan) | Source · checked · prefill 271 |
| NVIDIA GB10 | no speculative decoding | 30.8 | vLLM 0.31.0, 128K ctx, 1 user | Source · checked |