Models / Qwen
Can an RTX 4090 run Qwen3.6 35B-A3B?
MoE with 35B stored and 3B active. Faster than the parameter total suggests; memory still holds every expert.
- Stored parameters
- 35B
- Active parameters
- 3B
- Q4 total (estimate)
- 22.1 GB
- License
- See model card
Model card. Qwen's card: 35B total parameters, 3B activated, 262,144-token native context.
Compatibility
| Hardware | Q4 | Q8 | FP16 |
|---|---|---|---|
| M5 Ultra 256GB Apple · 256 GB unified
| 38.1 GB Fits estimate · KV at 8k No measured run | 70.9 GB Fits estimate · KV at 8k No measured run | |
| Ryzen AI Max+ 395 128GB AMD · 128 GB unified | 70.9 GB Fits estimate · KV at 8k No measured run | ||
| M5 Max 128GB Apple · 128 GB unified | 38.1 GB Fits estimate · KV at 8k No measured run | 70.9 GB Fits estimate · KV at 8k No measured run | |
| 4× RTX 3090 NVIDIA · 96 GB VRAM | 38.1 GB Fits estimate · KV at 8k No measured run | 70.9 GB Fits estimate · KV at 8k No measured run | |
| Ryzen AI Max+ 395 64GB AMD · 64 GB unified | 38.1 GB Fits estimate · KV at 8k No measured run | 70.9 GB Over estimate · KV at 8k No measured run | |
| M5 Max 36GB Apple · 36 GB unified | 38.1 GB Over estimate · KV at 8k No measured run | 70.9 GB Over estimate · KV at 8k No measured run | |
| Intel Arc Pro B70 Intel · 32 GB VRAM | 38.1 GB Over estimate · KV at 8k No measured run | 70.9 GB Over estimate · KV at 8k No measured run | |
| M6 32GB Apple · 32 GB unified | 38.1 GB Over estimate · KV at 8k No measured run | 70.9 GB Over estimate · KV at 8k No measured run | |
| RTX 3090 NVIDIA · 24 GB VRAM | 38.1 GB Over estimate · KV at 8k No measured run | 70.9 GB Over estimate · KV at 8k No measured run | |
| RTX 4090 NVIDIA · 24 GB VRAM | 38.1 GB Over estimate · KV at 8k No measured run | 70.9 GB Over estimate · KV at 8k No measured run | |
| TITAN RTX NVIDIA · 24 GB VRAM | 38.1 GB Over estimate · KV at 8k No measured run | 70.9 GB Over estimate · KV at 8k No measured run | |
| Radeon RX 7900 XTX AMD · 24 GB VRAM | 38.1 GB Over estimate · KV at 8k No measured run | 70.9 GB Over estimate · KV at 8k No measured run | |
| M3 Ultra 512GB Apple · 512 GB unified | 22.1 GB Fits estimate · KV at 8k No measured run | 38.1 GB Fits estimate · KV at 8k No measured run | 70.9 GB Fits estimate · KV at 8k No measured run |
| 2× NVIDIA GB10 NVIDIA · 256 GB unified | 22.1 GB Fits estimate · KV at 8k No measured run | 38.1 GB Fits estimate · KV at 8k No measured run | 70.9 GB Fits estimate · KV at 8k No measured run |
| M3 Ultra 256GB Apple · 256 GB unified | 22.1 GB Fits estimate · KV at 8k No measured run | 38.1 GB Fits estimate · KV at 8k No measured run | 70.9 GB Fits estimate · KV at 8k No measured run |
| M2 Ultra 192GB Apple · 192 GB unified | 22.1 GB Fits estimate · KV at 8k No measured run | 38.1 GB Fits estimate · KV at 8k No measured run | 70.9 GB Fits estimate · KV at 8k No measured run |
| NVIDIA GB10 NVIDIA · 128 GB unified | 22.1 GB Fits estimate · KV at 8k No measured run | 38.1 GB Fits estimate · KV at 8k No measured run | 70.9 GB Fits estimate · KV at 8k No measured run |
| M1 Ultra 128GB Apple · 128 GB unified | 22.1 GB Fits estimate · KV at 8k No measured run | 38.1 GB Fits estimate · KV at 8k No measured run | 70.9 GB Fits estimate · KV at 8k No measured run |
| RTX Pro 6000 Blackwell NVIDIA · 96 GB VRAM
| 22.1 GB Fits estimate · KV at 8k No measured run | 38.1 GB Fits estimate · KV at 8k No measured run | 70.9 GB Fits estimate · KV at 8k No measured run |
| M5 Ultra 96GB Apple · 96 GB unified | 22.1 GB Fits estimate · KV at 8k No measured run | 38.1 GB Fits estimate · KV at 8k No measured run | 70.9 GB Tight estimate · KV at 8k No measured run |
| 2× RTX 3090 NVIDIA · 48 GB VRAM | 22.1 GB Fits estimate · KV at 8k No measured run | 38.1 GB Fits estimate · KV at 8k No measured run | 70.9 GB Over estimate · KV at 8k No measured run |
| M4 Pro 48GB Apple · 48 GB unified | 22.1 GB Fits estimate · KV at 8k No measured run | 38.1 GB Tight estimate · KV at 8k No measured run | 70.9 GB Over estimate · KV at 8k No measured run |
| M4 Max 36GB Apple · 36 GB unified | 22.1 GB Fits estimate · KV at 8k No measured run | 38.1 GB Over estimate · KV at 8k No measured run | 70.9 GB Over estimate · KV at 8k No measured run |
| RTX 5090 NVIDIA · 32 GB VRAM | 22.1 GB Fits estimate · KV at 8k No measured run | 38.1 GB Over estimate · KV at 8k No measured run | 70.9 GB Over estimate · KV at 8k No measured run |
| Radeon AI Pro R9700 AMD · 32 GB VRAM | 22.1 GB Fits estimate · KV at 8k No measured run | 38.1 GB Over estimate · KV at 8k No measured run | 70.9 GB Over estimate · KV at 8k No measured run |
| M4 32GB Apple · 32 GB unified | 22.1 GB Fits estimate · KV at 8k No measured run | 38.1 GB Over estimate · KV at 8k No measured run | 70.9 GB Over estimate · KV at 8k No measured run |
| M4 Pro 24GB Apple · 24 GB unified | 22.1 GB Tight estimate · KV at 8k No measured run | 38.1 GB Over estimate · KV at 8k No measured run | 70.9 GB Over estimate · KV at 8k No measured run |
| M3 Pro 18GB Apple · 18 GB unified | 22.1 GB Over estimate · KV at 8k No measured run | 38.1 GB Over estimate · KV at 8k No measured run | 70.9 GB Over estimate · KV at 8k No measured run |
| RTX 4060 Ti 16GB NVIDIA · 16 GB VRAM | 22.1 GB Over estimate · KV at 8k No measured run | 38.1 GB Over estimate · KV at 8k No measured run | 70.9 GB Over estimate · KV at 8k No measured run |
| RTX 5060 Ti 16GB NVIDIA · 16 GB VRAM | 22.1 GB Over estimate · KV at 8k No measured run | 38.1 GB Over estimate · KV at 8k No measured run | 70.9 GB Over estimate · KV at 8k No measured run |
| RTX 5070 Ti NVIDIA · 16 GB VRAM | 22.1 GB Over estimate · KV at 8k No measured run | 38.1 GB Over estimate · KV at 8k No measured run | 70.9 GB Over estimate · KV at 8k No measured run |
| M4 16GB Apple · 16 GB unified | 22.1 GB Over estimate · KV at 8k No measured run | 38.1 GB Over estimate · KV at 8k No measured run | 70.9 GB Over estimate · KV at 8k No measured run |
| RTX 3060 12GB NVIDIA · 12 GB VRAM | 22.1 GB Over estimate · KV at 8k No measured run | 38.1 GB Over estimate · KV at 8k No measured run | 70.9 GB Over estimate · KV at 8k No measured run |
| RTX 3080 Ti NVIDIA · 12 GB VRAM | 22.1 GB Over estimate · KV at 8k No measured run | 38.1 GB Over estimate · KV at 8k No measured run | 70.9 GB Over estimate · KV at 8k No measured run |
| RTX 4070 Ti NVIDIA · 12 GB VRAM | 22.1 GB Over estimate · KV at 8k No measured run | 38.1 GB Over estimate · KV at 8k No measured run | 70.9 GB Over estimate · KV at 8k No measured run |
| RTX 2060 Super NVIDIA · 8 GB VRAM | 22.1 GB Over estimate · KV at 8k No measured run | 38.1 GB Over estimate · KV at 8k No measured run | 70.9 GB Over estimate · KV at 8k No measured run |
Measured runs
| Hardware | Quant | tok/s | Runtime | Source |
|---|---|---|---|---|
| M5 Max 128GB | 4-bit MLX (mlx-serve build) | 271 · speculative | mlx-serve 26.10.2 + MTP | Source · checked |
| RTX Pro 6000 Blackwell | NVFP4 | 261 | vLLM, b12x FP4 MoE kernel | Source · checked · prefill 26600 |
| RTX 5090 | llama-bench tg128 (quant per result) | 258 | llama.cpp | Source · checked |
| M5 Ultra 96GB | Ollama qwen3.6:35b-a3b default tag (23 GB) | 246 | Ollama 0.40.13 (llamacpp runner) | Source · checked |
| M5 Max 128GB | 4-bit MLX | 240 · speculative | Rapid-MLX 0.16.1 + MTP | Source · checked |
| 4× RTX 3090 | AWQ-INT4 + MTP-3 (TP=2) | 225–231 · speculative | vLLM | Source · checked |
| TITAN RTX | Unsloth UD-Q4_K_M with embedded MTP head (21.1 GiB), q8_0 KV, 65k context | 176.4 · speculative | llama.cpp b11222 (CUDA), llama-server, MTP draft 2 | Source · checked |
| M5 Max 36GB | 4-bit MLX | 173.4 · speculative | Rapid-MLX 0.16.1 (MTP) | Source · checked |
| RTX 3090 | Q4_K_M | 172.4 | llama.cpp CUDA b11804 | Source · checked · prefill 2763 |
| RTX 3090 | Q4_K_M | 162.24 | llama.cpp v0.7.0 llama-bench (CUDA), fully on GPU | Source · checked · prefill 4362 |
| RTX 4090 | Q4_K_M | 160.6 | llama.cpp b8931 | Source · checked · prefill 4589 |
| M5 Ultra 256GB | MLX affine 4-bit | 157 | MLX | Source · checked · prefill 8700 |
| 4× RTX 3090 | AWQ-INT4 (TP=2) | 149 | vLLM | Source · checked |
| RTX 3090 | llama-bench tg128 (quant per result) | 146–153 | llama.cpp | Source · checked |
| M5 Max 36GB | 4-bit MLX | 145.8 | oMLX 0.7.0 | Source · checked |
| M5 Ultra 256GB | MLX NVFP4 | 142 | MLX | Source · checked |
| RTX 3090 | Q4 GGUF, all layers on GPU, ~90K ctx | 139.6 | llama.cpp CUDA | Source · checked · prefill 3360 |
| Radeon RX 7900 XTX | UD-Q4_K_XL + MTP (draft-n-max 4), ~16K context | 135.2 · speculative | llama.cpp ROCm (llama-server) | Source · checked · prefill 1817 |
| M5 Ultra 256GB | GGUF Unsloth UD-Q4_K_XL | 134 | llama.cpp | Source · checked |
| M5 Max 36GB | 4-bit MLX | 124.5 | LM Studio (MLX) | Source · checked |
| Intel Arc Pro B70 | GPTQ Int4 | 120 | vLLM XPU b70 image, no MTP | Source · checked · prefill 9300 |
| M5 Ultra 256GB | MLX MXFP8 | 118 | MLX | Source · checked |
| 4× RTX 3090 | IQ4_XS (1 GPU) | 115–133 | llama.cpp | Source · checked |
| TITAN RTX | Unsloth UD-Q4_K_M with embedded MTP head (21.1 GiB), q8_0 KV, 65k context | 113.4 | llama.cpp b11222 (CUDA) | Source · checked · prefill 2180 |
| NVIDIA GB10 | NVFP4 | 113 · speculative | vLLM 0.30 + MTP | Source · checked · prefill 5860 |
| Radeon RX 7900 XTX | UD-Q4_K_XL + MTP, ~64K context | 108.2 · speculative | llama.cpp ROCm (llama-server) | Source · checked · prefill 1289 |
| M5 Max 128GB | 4-bit MLX | 107 | mlx_lm.server 0.32.2 | Source · checked |
| NVIDIA GB10 | NVFP4 + MTP (k=2) | 102.3 · speculative | vLLM 0.31.0, 128K ctx, 1 user | Source · checked |
| NVIDIA GB10 | NVFP4 + MTP | 95.1 · speculative | vLLM, 1 stream | Source · checked |
| Ryzen AI Max+ 395 128GB | Q4_K_M (Unsloth) | 93 · speculative | llama.cpp Vulkan + MTP | Source · checked |
| Radeon RX 7900 XTX | UD-Q4_K_XL, llama-bench 2K, f16 KV | 87.2 | llama.cpp ROCm | Source · checked · prefill 2724 |
| NVIDIA GB10 | NVFP4 | 86 · speculative | vLLM 0.30 + MTP | Source · checked · prefill 5860 |
| Ryzen AI Max+ 395 128GB | Unsloth UD-Q4_K_M with embedded MTP head (21.1 GiB), q8_0 KV, 65k context | 81.9 · speculative | llama.cpp (Vulkan), llama-server, MTP draft 3 | Source · checked |
| Radeon RX 7900 XTX | UD-Q4_K_XL, ~16K real-text context, q4_0 KV | 79.6 | llama.cpp ROCm (llama-server) | Source · checked · prefill 1918 |
| NVIDIA GB10 | NVFP4, no speculative decoding | 78.1 | vLLM 0.31.0, 128K ctx, 1 user | Source · checked |
| RTX 3090 | Q4_K_M, 20 layers of experts in system RAM (-ncmoe 20) | 73.59 | llama.cpp v0.7.0 llama-server + --moe-cache-mib 1866 | Source · checked · prefill 301 |
| M6 32GB | MLX affine 4-bit (group 64) + MTP head and DFlash2 drafter | 69.3 · speculative | Kapitsa 0.2.4 (KFlash tree speculative decoding) | Source · checked |
| Ryzen AI Max+ 395 64GB | UD-Q4_K_M (22.7 GB) + MTP draft N=2 | 68.03 · speculative | llama.cpp b11065 (ROCm) | Source · checked · prefill 1240 |
| Radeon RX 7900 XTX | UD-Q4_K_XL, ~64K context | 66.0 | llama.cpp ROCm (llama-server) | Source · checked · prefill 1368 |
| M6 32GB | MLX 4-bit | 63.2 | LM Studio (MLX) | Source · checked |
| Ryzen AI Max+ 395 128GB | Q4_K_M (Unsloth) | 62 | llama.cpp Vulkan | Source · checked · prefill 1380 |
| Ryzen AI Max+ 395 128GB | Unsloth UD-Q4_K_XL (22.4 GB) | 61.8 | llama.cpp b11401 (Windows x64 Vulkan, unmodified) | Source · checked |
| Ryzen AI Max+ 395 128GB | Q8_0 + MTP (draft-n-max 3), coding workload | 60.92 · speculative | llama.cpp ROCm | Source · checked |
| Ryzen AI Max+ 395 128GB | Q4_K_M | 59.4 | llama.cpp b11213 Vulkan | Source · checked |
| Ryzen AI Max+ 395 128GB | Unsloth UD-Q4_K_M with embedded MTP head (21.1 GiB), q8_0 KV, 65k context | 57.4 | llama.cpp (Vulkan), llama-server | Source · checked |
| Ryzen AI Max+ 395 128GB | 4-bit | 54.4 | Ollama 0.34.0 | Source · checked |
| Ryzen AI Max+ 395 128GB | Q8_0, llama-bench tg128 | 54.22 | llama.cpp Vulkan | Source · checked · prefill 911 |
| Ryzen AI Max+ 395 128GB | Q8_0 | 54 | llama.cpp Vulkan | Source · checked · prefill 1370 |
| Ryzen AI Max+ 395 64GB | UD-Q4_K_M (22.1 GB) | 52.3 | llama.cpp b11669 (Vulkan) | Source · checked · prefill 1006 |
| M6 32GB | Splash package (incoai/Qwen3.6-35B-A3B-Splash) | 51.5 · speculative | LM Studio Splash backend | Source · checked |
| Ryzen AI Max+ 395 128GB | Q4_K_M (Unsloth) | 51 | llama.cpp ROCm | Source · checked · prefill 1310 |
| Ryzen AI Max+ 395 128GB | Q8_0, llama-bench tg128 | 47.47 | llama.cpp ROCm | Source · checked · prefill 925 |
| Radeon RX 7900 XTX | UD-Q4_K_XL, ~245K context, q4_0 KV | 38.3 | llama.cpp ROCm (llama-server) | Source · checked · prefill 641 |
| Ryzen AI Max+ 395 64GB | UD-Q4_K_M (22.7 GB) + MTP draft N=2 | 21.66 · speculative | llama.cpp b11065 (ROCm) | Source · checked |