Skip to content
RigCheck

Models / Llama

Can 2× RTX 3090 run Llama 3.3 70B?

Dense 70B instruct model. The usual 'will this fit on 24 GB?' question.

Stored parameters
70B
Active parameters
Dense
Q4 total (estimate)
45.7 GB
License
Llama 3.3 Community License

Model card. Meta's model card lists Llama 3.3 as a 70B text model with grouped-query attention.

Compatibility

HardwareQ4Q8FP16
M5 Ultra 256GB
Apple · 256 GB unified
45.7 GB
Fits
estimate · KV at 8k
Measured · 25.2 tok/s median
77.6 GB
Fits
estimate · KV at 8k
No measured run
143 GB
Fits
estimate · KV at 8k
No measured run
2× RTX 3090
NVIDIA · 48 GB VRAM
45.7 GB
Tight
estimate · KV at 8k
Measured · 19.1 tok/s median
77.6 GB
Over
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run
M3 Ultra 512GB
Apple · 512 GB unified
45.7 GB
Fits
estimate · KV at 8k
No measured run
77.6 GB
Fits
estimate · KV at 8k
No measured run
143 GB
Fits
estimate · KV at 8k
No measured run
2× NVIDIA GB10
NVIDIA · 256 GB unified
45.7 GB
Fits
estimate · KV at 8k
No measured run
77.6 GB
Fits
estimate · KV at 8k
No measured run
143 GB
Fits
estimate · KV at 8k
No measured run
M3 Ultra 256GB
Apple · 256 GB unified
45.7 GB
Fits
estimate · KV at 8k
No measured run
77.6 GB
Fits
estimate · KV at 8k
No measured run
143 GB
Fits
estimate · KV at 8k
No measured run
M2 Ultra 192GB
Apple · 192 GB unified
45.7 GB
Fits
estimate · KV at 8k
No measured run
77.6 GB
Fits
estimate · KV at 8k
No measured run
143 GB
Tight
estimate · KV at 8k
No measured run
NVIDIA GB10
NVIDIA · 128 GB unified
45.7 GB
Fits
estimate · KV at 8k
No measured run
77.6 GB
Fits
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run
Ryzen AI Max+ 395 128GB
AMD · 128 GB unified
45.7 GB
Fits
estimate · KV at 8k
No measured run
77.6 GB
Fits
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run
M1 Ultra 128GB
Apple · 128 GB unified
45.7 GB
Fits
estimate · KV at 8k
No measured run
77.6 GB
Fits
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run
M5 Max 128GB
Apple · 128 GB unified
45.7 GB
Fits
estimate · KV at 8k
No measured run
77.6 GB
Fits
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run
4× RTX 3090
NVIDIA · 96 GB VRAM
45.7 GB
Fits
estimate · KV at 8k
No measured run
77.6 GB
Fits
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run
RTX Pro 6000 Blackwell
NVIDIA · 96 GB VRAM
45.7 GB
Fits
estimate · KV at 8k
No measured run
77.6 GB
Fits
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run
M5 Ultra 96GB
Apple · 96 GB unified
45.7 GB
Fits
estimate · KV at 8k
No measured run
77.6 GB
Tight
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run
Ryzen AI Max+ 395 64GB
AMD · 64 GB unified
45.7 GB
Tight
estimate · KV at 8k
No measured run
77.6 GB
Over
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run
M4 Pro 48GB
Apple · 48 GB unified
45.7 GB
Tight
estimate · KV at 8k
No measured run
77.6 GB
Over
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run
M4 Max 36GB
Apple · 36 GB unified
45.7 GB
Over
estimate · KV at 8k
No measured run
77.6 GB
Over
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run
M5 Max 36GB
Apple · 36 GB unified
45.7 GB
Over
estimate · KV at 8k
No measured run
77.6 GB
Over
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run
RTX 5090
NVIDIA · 32 GB VRAM
45.7 GB
Over
estimate · KV at 8k
No measured run
77.6 GB
Over
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run
Radeon AI Pro R9700
AMD · 32 GB VRAM
45.7 GB
Over
estimate · KV at 8k
No measured run
77.6 GB
Over
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run
Intel Arc Pro B70
Intel · 32 GB VRAM
45.7 GB
Over
estimate · KV at 8k
No measured run
77.6 GB
Over
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run
M4 32GB
Apple · 32 GB unified
45.7 GB
Over
estimate · KV at 8k
No measured run
77.6 GB
Over
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run
M6 32GB
Apple · 32 GB unified
45.7 GB
Over
estimate · KV at 8k
No measured run
77.6 GB
Over
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run
RTX 3090
NVIDIA · 24 GB VRAM
45.7 GB
Over
estimate · KV at 8k
No measured run
77.6 GB
Over
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run
RTX 4090
NVIDIA · 24 GB VRAM
45.7 GB
Over
estimate · KV at 8k
No measured run
77.6 GB
Over
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run
TITAN RTX
NVIDIA · 24 GB VRAM
45.7 GB
Over
estimate · KV at 8k
No measured run
77.6 GB
Over
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run
Radeon RX 7900 XTX
AMD · 24 GB VRAM
45.7 GB
Over
estimate · KV at 8k
No measured run
77.6 GB
Over
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run
M4 Pro 24GB
Apple · 24 GB unified
45.7 GB
Over
estimate · KV at 8k
No measured run
77.6 GB
Over
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run
M3 Pro 18GB
Apple · 18 GB unified
45.7 GB
Over
estimate · KV at 8k
No measured run
77.6 GB
Over
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run
RTX 4060 Ti 16GB
NVIDIA · 16 GB VRAM
45.7 GB
Over
estimate · KV at 8k
No measured run
77.6 GB
Over
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run
RTX 5060 Ti 16GB
NVIDIA · 16 GB VRAM
45.7 GB
Over
estimate · KV at 8k
No measured run
77.6 GB
Over
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run
RTX 5070 Ti
NVIDIA · 16 GB VRAM
45.7 GB
Over
estimate · KV at 8k
No measured run
77.6 GB
Over
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run
M4 16GB
Apple · 16 GB unified
45.7 GB
Over
estimate · KV at 8k
No measured run
77.6 GB
Over
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run
RTX 3060 12GB
NVIDIA · 12 GB VRAM
45.7 GB
Over
estimate · KV at 8k
No measured run
77.6 GB
Over
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run
RTX 3080 Ti
NVIDIA · 12 GB VRAM
45.7 GB
Over
estimate · KV at 8k
No measured run
77.6 GB
Over
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run
RTX 4070 Ti
NVIDIA · 12 GB VRAM
45.7 GB
Over
estimate · KV at 8k
No measured run
77.6 GB
Over
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run
RTX 2060 Super
NVIDIA · 8 GB VRAM
45.7 GB
Over
estimate · KV at 8k
No measured run
77.6 GB
Over
estimate · KV at 8k
No measured run
143 GB
Over
estimate · KV at 8k
No measured run

Measured runs

HardwareQuanttok/sRuntimeSource
M5 Ultra 256GB4-bit MLX (mlx-community)25.2mlx-lm 0.32.2 (MLX 0.32.4)Source · checked · prefill 682
2× RTX 3090not stated20ExLlamaV2 (TP)Source
2× RTX 3090Q4_K_M19.1llama.cpp 0.7.0-devSource · checked · prefill 939
RTX 4060 Ti 16GBQ5_K_M1.5LM Studio / KoboldSource