Models / Qwen
Can an RTX 4090 run Qwen3.8 27B?
Dense 27B vision-language model. The most-measured model in the current RigBase import.
- Stored parameters
- 27B
- Active parameters
- Dense
- Q4 total (estimate)
- 17.6 GB
- License
- See model card
Model card. Qwen's card lists 27B parameters and a 262,144-token native context.
Compatibility
| Hardware | Q4 | Q8 | FP16 |
|---|---|---|---|
| Ryzen AI Max+ 395 128GB AMD · 128 GB unified
| 55.3 GB Fits estimate · KV at 8k No measured run | ||
| M5 Max 128GB Apple · 128 GB unified | 29.9 GB Fits estimate · KV at 8k No measured run | 55.3 GB Fits estimate · KV at 8k No measured run | |
| Ryzen AI Max+ 395 64GB AMD · 64 GB unified | 29.9 GB Fits estimate · KV at 8k No measured run | 55.3 GB Tight estimate · KV at 8k No measured run | |
| 2× RTX 3090 NVIDIA · 48 GB VRAM
| 29.9 GB Fits estimate · KV at 8k No measured run | 55.3 GB Over estimate · KV at 8k No measured run | |
| M5 Max 36GB Apple · 36 GB unified | 29.9 GB Tight estimate · KV at 8k No measured run | 55.3 GB Over estimate · KV at 8k No measured run | |
| RTX 5090 NVIDIA · 32 GB VRAM | 29.9 GB Tight estimate · KV at 8k No measured run | 55.3 GB Over estimate · KV at 8k No measured run | |
| Radeon AI Pro R9700 AMD · 32 GB VRAM
| 29.9 GB Tight estimate · KV at 8k No measured run | 55.3 GB Over estimate · KV at 8k No measured run | |
| Intel Arc Pro B70 Intel · 32 GB VRAM | 29.9 GB Tight estimate · KV at 8k No measured run | 55.3 GB Over estimate · KV at 8k No measured run | |
| M6 32GB Apple · 32 GB unified | 29.9 GB Tight estimate · KV at 8k No measured run | 55.3 GB Over estimate · KV at 8k No measured run | |
| RTX 3090 NVIDIA · 24 GB VRAM | 29.9 GB Over estimate · KV at 8k No measured run | 55.3 GB Over estimate · KV at 8k No measured run | |
| RTX 4090 NVIDIA · 24 GB VRAM | 29.9 GB Over estimate · KV at 8k No measured run | 55.3 GB Over estimate · KV at 8k No measured run | |
| TITAN RTX NVIDIA · 24 GB VRAM | 29.9 GB Over estimate · KV at 8k No measured run | 55.3 GB Over estimate · KV at 8k No measured run | |
| RTX 5060 Ti 16GB NVIDIA · 16 GB VRAM | 29.9 GB Over estimate · KV at 8k No measured run | 55.3 GB Over estimate · KV at 8k No measured run | |
| M3 Ultra 512GB Apple · 512 GB unified | 17.6 GB Fits estimate · KV at 8k No measured run | 29.9 GB Fits estimate · KV at 8k No measured run | |
| 2× NVIDIA GB10 NVIDIA · 256 GB unified | 17.6 GB Fits estimate · KV at 8k No measured run | 29.9 GB Fits estimate · KV at 8k No measured run | 55.3 GB Fits estimate · KV at 8k No measured run |
| M3 Ultra 256GB Apple · 256 GB unified | 17.6 GB Fits estimate · KV at 8k No measured run | 29.9 GB Fits estimate · KV at 8k No measured run | 55.3 GB Fits estimate · KV at 8k No measured run |
| M5 Ultra 256GB Apple · 256 GB unified | 17.6 GB Fits estimate · KV at 8k No measured run | 29.9 GB Fits estimate · KV at 8k No measured run | 55.3 GB Fits estimate · KV at 8k No measured run |
| M2 Ultra 192GB Apple · 192 GB unified | 17.6 GB Fits estimate · KV at 8k No measured run | 29.9 GB Fits estimate · KV at 8k No measured run | 55.3 GB Fits estimate · KV at 8k No measured run |
| NVIDIA GB10 NVIDIA · 128 GB unified | 17.6 GB Fits estimate · KV at 8k No measured run | 29.9 GB Fits estimate · KV at 8k No measured run | 55.3 GB Fits estimate · KV at 8k No measured run |
| M1 Ultra 128GB Apple · 128 GB unified | 17.6 GB Fits estimate · KV at 8k No measured run | 29.9 GB Fits estimate · KV at 8k No measured run | 55.3 GB Fits estimate · KV at 8k No measured run |
| 4× RTX 3090 NVIDIA · 96 GB VRAM | 17.6 GB Fits estimate · KV at 8k No measured run | 29.9 GB Fits estimate · KV at 8k No measured run | 55.3 GB Fits estimate · KV at 8k No measured run |
| RTX Pro 6000 Blackwell NVIDIA · 96 GB VRAM | 17.6 GB Fits estimate · KV at 8k No measured run | 29.9 GB Fits estimate · KV at 8k No measured run | 55.3 GB Fits estimate · KV at 8k No measured run |
| M5 Ultra 96GB Apple · 96 GB unified
| 17.6 GB Fits estimate · KV at 8k No measured run | 29.9 GB Fits estimate · KV at 8k No measured run | 55.3 GB Fits estimate · KV at 8k No measured run |
| M4 Pro 48GB Apple · 48 GB unified | 17.6 GB Fits estimate · KV at 8k No measured run | 29.9 GB Fits estimate · KV at 8k No measured run | 55.3 GB Over estimate · KV at 8k No measured run |
| M4 Max 36GB Apple · 36 GB unified | 17.6 GB Fits estimate · KV at 8k No measured run | 29.9 GB Tight estimate · KV at 8k No measured run | 55.3 GB Over estimate · KV at 8k No measured run |
| M4 32GB Apple · 32 GB unified | 17.6 GB Fits estimate · KV at 8k No measured run | 29.9 GB Tight estimate · KV at 8k No measured run | 55.3 GB Over estimate · KV at 8k No measured run |
| Radeon RX 7900 XTX AMD · 24 GB VRAM | 17.6 GB Fits estimate · KV at 8k No measured run | 29.9 GB Over estimate · KV at 8k No measured run | 55.3 GB Over estimate · KV at 8k No measured run |
| M4 Pro 24GB Apple · 24 GB unified
| 17.6 GB Tight estimate · KV at 8k No measured run | 29.9 GB Over estimate · KV at 8k No measured run | 55.3 GB Over estimate · KV at 8k No measured run |
| M3 Pro 18GB Apple · 18 GB unified | 17.6 GB Tight estimate · KV at 8k No measured run | 29.9 GB Over estimate · KV at 8k No measured run | 55.3 GB Over estimate · KV at 8k No measured run |
| RTX 4060 Ti 16GB NVIDIA · 16 GB VRAM | 17.6 GB Over estimate · KV at 8k No measured run | 29.9 GB Over estimate · KV at 8k No measured run | 55.3 GB Over estimate · KV at 8k No measured run |
| RTX 5070 Ti NVIDIA · 16 GB VRAM | 17.6 GB Over estimate · KV at 8k No measured run | 29.9 GB Over estimate · KV at 8k No measured run | 55.3 GB Over estimate · KV at 8k No measured run |
| M4 16GB Apple · 16 GB unified | 17.6 GB Over estimate · KV at 8k No measured run | 29.9 GB Over estimate · KV at 8k No measured run | 55.3 GB Over estimate · KV at 8k No measured run |
| RTX 3060 12GB NVIDIA · 12 GB VRAM | 17.6 GB Over estimate · KV at 8k No measured run | 29.9 GB Over estimate · KV at 8k No measured run | 55.3 GB Over estimate · KV at 8k No measured run |
| RTX 3080 Ti NVIDIA · 12 GB VRAM | 17.6 GB Over estimate · KV at 8k No measured run | 29.9 GB Over estimate · KV at 8k No measured run | 55.3 GB Over estimate · KV at 8k No measured run |
| RTX 4070 Ti NVIDIA · 12 GB VRAM | 17.6 GB Over estimate · KV at 8k No measured run | 29.9 GB Over estimate · KV at 8k No measured run | 55.3 GB Over estimate · KV at 8k No measured run |
| RTX 2060 Super NVIDIA · 8 GB VRAM | 17.6 GB Over estimate · KV at 8k No measured run | 29.9 GB Over estimate · KV at 8k No measured run | 55.3 GB Over estimate · KV at 8k No measured run |
Measured runs
| Hardware | Quant | tok/s | Runtime | Source |
|---|---|---|---|---|
| RTX 5090 | all-NVFP4 NInfer v3 pack of the stock model (kaushikvira/Qwen3.8-27B-nvfp4full-dflash2-NInfer-v3) | 400 · speculative | NInfer (ninfer-all fork) + DFlash2 drafter, 7 draft tokens, INT8 KV | Source · checked · prefill 11181 |
| RTX 5090 | all-NVFP4 NInfer v3 pack of the stock model (kaushikvira/Qwen3.8-27B-nvfp4full-dflash2-NInfer-v3) | 326 · speculative | NInfer (ninfer-all fork) + DFlash2 drafter, 7 draft tokens, INT8 KV | Source · checked · prefill 7856 |
| RTX 5090 | all-NVFP4 NInfer v3 pack of the stock model (kaushikvira/Qwen3.8-27B-nvfp4full-dflash2-NInfer-v3) | 165 · speculative | NInfer (ninfer-all fork) + DFlash2 drafter, 7 draft tokens, INT8 KV | Source · checked · prefill 12908 |
| RTX Pro 6000 Blackwell | NVFP4 (W4A16) + MTP (5 draft tokens), 256K context | 150 · speculative | vLLM 0.27.1 | Source · checked · prefill 3800 |
| RTX 3090 | Q4_K_M (unsloth), greedy code prompt | 139.7 · speculative | OpenJet megakernel (MTP, 4 drafts) | Source · checked |
| RTX 5090 | Q4_K_M, q4_0 KV | 136.7 · speculative | llama.cpp b10448, MTP n=2 (--spec-type draft-mtp) | Source · checked |
| RTX 3090 | AutoRound W4A16 | 129.8 · speculative | vLLM (patched) | Source · checked |
| 2× RTX 3090 | INT4 (RedHat INT4 checkpoint, QEngine layout), fp8 KV | 127.5 · speculative | QEngine 1.3.2 (own collectives, P2P over NVLink), MTP head | Source · checked · prefill 2363 |
| RTX 5090 | Ollama qwen3.8:27b q4 (17 GB on disk) | 124 | Ollama 0.32.13 | Source · checked |
| Intel Arc Pro B70 | GPTQ Int4 (asym) | 96 · speculative | vLLM XPU b70 image + MTP, reasoning prompt | Source · checked · prefill 2042 |
| 2× RTX 3090 | W4A16 AutoRound + DFlash2 | 91.2 · speculative | vLLM TP2, 1 stream | Source · checked |
| RTX Pro 6000 Blackwell | BF16 + MTP (5 draft tokens), 256K context | 85 · speculative | vLLM 0.20.1 | Source · checked · prefill 2400 |
| RTX 3090 | Q4_K_M (unsloth), reasoning mode | 78.0 · speculative | OpenJet megakernel (MTP) | Source · checked |
| RTX 5090 | Q4_K_M (EPYC 9534 host) | 75.2 | llama.cpp 0.7.0-dev (b11669) | Source · checked · prefill 2500 |
| RTX 5090 | Q4_K_M | 75.19 | llama.cpp | Source · checked · prefill 2500 |
| RTX 5090 | Q4_K_M, f16 KV | 73.6 | llama.cpp b10448, MTP off | Source · checked · prefill 3036 |
| RTX 3090 | Q4_K_M (unsloth), greedy code prompt | 72.5 · speculative | llama.cpp (MTP, 2 drafts) | Source · checked |
| 2× RTX 3090 | AWQ + DSpark | 69.9 · speculative | SGLang TP2, 1 stream | Source · checked |
| Intel Arc Pro B70 | GPTQ Int4 (asym) | 66 · speculative | vLLM XPU b70 image + MTP, prose prompt | Source · checked · prefill 2042 |
| RTX 4090 | Q4_K_M (16.8 GB, LM Studio catalogue) + the model's own MTP (LM Studio default) | 62.4 · speculative | LM Studio 2.55.0 (llama.cpp CUDA) | Source · checked · prefill 1994 |
| NVIDIA GB10 | MLX 4-bit + DFlash2 | 58.3 · speculative | TensorFold, 1 stream | Source · checked · prefill 1633 |
| RTX 3090 | Unsloth UD-Q4_K_M (nextn/MTP head embedded), f16 KV | 56.37 · speculative | llama.cpp b11382 (Windows, CUDA 13), MTP n=2 | Source · checked · prefill 1073 |
| 2× RTX 3090 | Q6_K_L (bartowski GGUF) | 55.4 · speculative | llama.cpp b10666, -sm tensor (NCCL) | Source · checked · prefill 743 |
| RTX 3090 | Q4_K_M (unsloth), reasoning mode | 50.9 · speculative | llama.cpp (MTP, 2 drafts) | Source · checked |
| RTX 5060 Ti 16GB | Unsloth UD-IQ4_XS + mtp-Qwen3.8-27B-Q4_0 head, q8_0 KV | 50.2 · speculative | llama.cpp 23b0202a, MTP-2 | Source · checked |
| TITAN RTX | Unsloth UD-Q4_K_XL, q8_0 KV, 65k context (19.5 GiB) | 49.0 · speculative | llama.cpp b11222 (CUDA), llama-server, MTP draft 2 | Source · checked |
| 2× RTX 3090 | Q6_K_L (bartowski GGUF) | 48.2 | llama.cpp b10666, -sm tensor (NCCL) | Source · checked |
| M5 Ultra 96GB | MXFP8 (Ollama qwen3.8:27b-mxfp8, 31 GB) | 47.2 | Ollama 0.40.13 (MLX runner) | Source · checked · prefill 1376 |
| RTX 5060 Ti 16GB | Unsloth UD-IQ4_XS + mtp-Qwen3.8-27B-Q4_0 head, q8_0 KV | 45.8 · speculative | llama.cpp 23b0202a, MTP-1 | Source · checked |
| M5 Max 128GB | 4-bit MLX + Qwen3.8-27B-MTP-4bit drafter | 45.3 · speculative | oMLX v0.6.0, MTP enabled | Source · checked · prefill 782 |
| 2× NVIDIA GB10 | NVFP4 (Unsloth, 4-bit, 23.4 GB on disk) | 44.7 · speculative | vLLM 0.1.dev20003 TP2, MTP n=3, FP8 KV, 351K context | Source · checked · prefill 2034 |
| RTX 4090 | Q4_K_M | 43.1 | llama.cpp build 11803 (a37cc4610) | Source · checked · prefill 515 |
| RTX 4090 | Q4_K_M | 43.06 | llama.cpp (df18bee) | Source · checked · prefill 429 |
| RTX 4090 | Q4_K_M | 43.0 | llama.cpp build 11803 (a37cc4610) | Source · checked · prefill 2015 |
| Radeon AI Pro R9700 | IQ4_NL + MTP draft 2 | 43.0 · speculative | llama-server HIP, 4 slots, 131k unified context | Source · checked · prefill 979 |
| RTX 3090 | Q4_K_M (unsloth), no speculation | 42.9 | OpenJet megakernel | Source · checked |
| RTX 4090 | Q4_K_M | 42.63 | llama.cpp (df18bee) | Source · checked · prefill 1915 |
| RTX 3090 | UD-Q4_K_XL (GGUF) | 40.6 · speculative | llama.cpp, tuned | Source · checked |
| 2× RTX 3090 | UD-Q4_K_M (GGUF) | 40.556 | llama.cpp b10775, 2 GPU -sm layer | Source · checked · prefill 1616 |
| RTX 3090 | Q4_K_M (unsloth), no speculation | 40.4 | llama.cpp | Source · checked |
| 2× RTX 3090 | UD-Q4_K_M (GGUF) | 39.157 | llama.cpp b10775, 1 GPU | Source · checked · prefill 1235 |
| RTX 3090 | Q4_K_M | 39 | llama.cpp server | Source · checked · prefill 1226 |
| RTX 3090 | UD-Q4_K_XL (GGUF) | 38.8 | llama.cpp llama-bench, 1 GPU at 350 W | Source · checked · prefill 1027 |
| Ryzen AI Max+ 395 128GB | ROCmFP4-FAST (julianmb/Qwen-3.8-27B-ROCmFP4-FAST-GGUF), q8_0 KV | 38.6 · speculative | ROCmFPX llama.cpp fork main 120141f1f (HIP), --spec-type draft-mtp, 4 draft tokens | Source · checked · prefill 353 |
| RTX 3090 | Q4_K_M | 38.6 | llama.cpp build 11803 (a37cc4610) | Source · checked · prefill 483 |
| RTX 3090 | Unsloth UD-Q4_K_M, f16 KV | 38.3 | llama.cpp b11382 (Windows, CUDA 13), MTP off, -np 4 | Source · checked |
| RTX 3090 | llama-bench tg128 (quant per result) | 38.2–71.6 | llama.cpp | Source · checked |
| RTX 3090 | Q4_K_M | 38.1 | llama.cpp build 11803 (a37cc4610) | Source · checked · prefill 1209 |
| M5 Max 36GB | 4-bit MLX | 37.9 · speculative | Rapid-MLX 0.16.1 (MTP) | Source · checked |
| RTX 3090 | AutoRound W4A16 | 32.6 | vLLM (stock) | Source · checked |
| 2× RTX 3090 | Q6_K_L (bartowski GGUF) | 32.4 | llama.cpp b10666, -sm layer, out of the box | Source · checked |
| 2× RTX 3090 | UD-Q4_K_M (GGUF) | 31.252 | llama.cpp b10775, 2 GPU -sm tensor + NCCL 2.27.5 | Source · checked · prefill 169 |
| M5 Max 128GB | 4-bit MLX | 31 | same server, no speculation | Source · checked |
| TITAN RTX | Q4_K_M | 30.2 | llama.cpp b11222 (CUDA) | Source · checked · prefill 768 |
| Radeon AI Pro R9700 | IQ4_NL | 29.8 | llama.cpp HIP b10687 (llama-bench) | Source · checked · prefill 1287 |
| Ryzen AI Max+ 395 128GB | GSQ-RCO IQ3_S + MTP head (ISTA-DASLab GGUF, 12.12 GB) | 28.8 · speculative | llama.cpp 79e2e74eb (Vulkan) | Source · checked |
| Intel Arc Pro B70 | Q4_K_M | 27.8 | llama.cpp SYCL + neural.download TP1 stack, no speculation | Source · checked · prefill 825 |
| RTX 5060 Ti 16GB | Unsloth UD-IQ4_XS, q8_0 KV | 27.6 | llama.cpp 23b0202a, no speculative decoding | Source · checked |
| Radeon AI Pro R9700 | IQ4_NL, no speculation | 27.1 | llama-server HIP, 4 slots, 131k unified context | Source · checked · prefill 1031 |
| NVIDIA GB10 | NVFP4 (Unsloth, 4-bit, 23.4 GB on disk) | 26.7 · speculative | vLLM 0.1.dev20003, MTP n=3, FP8 KV, 351K context | Source · checked · prefill 1188 |
| Radeon AI Pro R9700 | UD-Q4_K_XL | 26.7 | llama.cpp HIP b10687 (llama-bench) | Source · checked · prefill 1151 |
| M5 Max 36GB | 4-bit MLX | 26.0 | LM Studio (MLX) | Source · checked |
| M5 Max 36GB | 4-bit MLX | 25.3 | oMLX 0.7.0 | Source · checked |
| Ryzen AI Max+ 395 64GB | Q4_K_M (17.1 GB, Unsloth) + MTP (2 draft tokens) | 25.0 · speculative | llama.cpp b11669 (Vulkan) | Source · checked · prefill 215 |
| Radeon AI Pro R9700 | UD-Q5_K_M | 24.5 | llama.cpp HIP b10687 (llama-bench) | Source · checked · prefill 946 |
| RTX 5060 Ti 16GB | Q4_0 | 24 | llama.cpp 65840ed (CUDA) | Source · checked · prefill 849 |
| Radeon AI Pro R9700 | UD-Q6_K | 23.2 | llama.cpp HIP b10687 (llama-bench) | Source · checked · prefill 821 |
| NVIDIA GB10 | + MTP (k=2) | 19.0 · speculative | vLLM 0.31.0, 128K ctx, 1 user | Source · checked |
| NVIDIA GB10 | NVFP4 + DSpark | 18.3 · speculative | SGLang, 1 stream | Source · checked |
| Ryzen AI Max+ 395 128GB | Q6_K + MTP (draft-n-max 2) | 18.01 · speculative | llama.cpp Vulkan (RADV) | Source · checked · prefill 23.9 |
| Ryzen AI Max+ 395 64GB | UD-Q4_K_XL (17.6 GB) + MTP draft N=4, KV q8_0 | 16.87 · speculative | llama.cpp (ROCm) | Source · checked · prefill 299 |
| Ryzen AI Max+ 395 128GB | GSQ-RCO IQ3_S + MTP head (ISTA-DASLab GGUF, 12.12 GB) | 16.1 | llama.cpp 79e2e74eb (Vulkan) | Source · checked |
| Ryzen AI Max+ 395 128GB | UD-Q3_K_XL (Unsloth) | 15.8 | llama.cpp (Vulkan) | Source · checked |
| Ryzen AI Max+ 395 128GB | Q8_0 + MTP (draft-n-max 3), coding workload, reasoning on | 15.67 · speculative | llama.cpp ROCm | Source · checked |
| M6 32GB | Splash package (incoai/Qwen3.8-27B-Splash) | 15.3 · speculative | LM Studio Splash backend | Source · checked |
| Ryzen AI Max+ 395 128GB | Q6_K + MTP (draft-n-max 5) | 15.16 · speculative | llama.cpp Vulkan (RADV) | Source · checked · prefill 23.4 |
| Ryzen AI Max+ 395 128GB | Q8_0 + MTP (draft-n-max 3), coding workload, reasoning on | 13.19 · speculative | llama.cpp Vulkan | Source · checked |
| Ryzen AI Max+ 395 128GB | UD-Q4_K_M (15.33 GiB) | 12.82 | llama.cpp (Vulkan, commit f5222f25c) | Source · checked · prefill 414 |
| Ryzen AI Max+ 395 128GB | Q4_K_M | 12.6 | llama.cpp/Ollama, ROCm | Source · checked |
| M3 Ultra 512GB | BF16, 8,192-token context | 12.46 | MLX (mlx_lm.benchmark) | Source · checked |
| Ryzen AI Max+ 395 64GB | Q4_K_M (17.1 GB, Unsloth) | 12.1 | llama.cpp b11669 (Vulkan) | Source · checked · prefill 205 |
| M4 Pro 24GB | GSQ-RCO IQ3_S + MTP head (ISTA-DASLab GGUF, 12.12 GB) | 11.4 | llama.cpp 0.6.0 (Metal) | Source · checked |
| M6 32GB | GGUF Q4_K_M | 11.2 | LM Studio (llama.cpp) | Source · checked |
| NVIDIA GB10 | no speculative decoding | 11.0 | vLLM 0.31.0, 128K ctx, 1 user | Source · checked |
| M4 Pro 24GB | GSQ-RCO IQ3_S + MTP head (ISTA-DASLab GGUF, 12.12 GB) | 10.9 · speculative | llama.cpp 0.6.0 (Metal) | Source · checked |
| RTX 4060 Ti 16GB | llama-bench tg128 (quant per result) | 10.7–17.6 | llama.cpp | Source · checked |
| Ryzen AI Max+ 395 128GB | UD-Q6_K (20.47 GiB) | 9.71 | llama.cpp (Vulkan, commit f5222f25c) | Source · checked · prefill 401 |
| Ryzen AI Max+ 395 128GB | Q6_K, no MTP | 9.59 | llama.cpp Vulkan (RADV) | Source · checked · prefill 26.0 |
| Ryzen AI Max+ 395 128GB | Q8_0, llama-bench tg128 | 7.77 | llama.cpp Vulkan | Source · checked · prefill 254 |
| Ryzen AI Max+ 395 128GB | Q8_0, llama-bench tg128 | 7.76 | llama.cpp ROCm | Source · checked · prefill 304 |
| Ryzen AI Max+ 395 128GB | Q8_0, --reasoning off | 7.7 | llama.cpp/Ollama, ROCm | Source · checked |
| Ryzen AI Max+ 395 128GB | Q8_0 (27.05 GiB) | 7.5 | llama.cpp (Vulkan, commit f5222f25c) | Source · checked · prefill 477 |
| RTX 3060 12GB | llama-bench tg128 (quant per result) | 5.2–6.7 | llama.cpp | Source · checked |