Skip to content
RigCheck

Models / Llama

Can an RTX 3060 12GB run Llama 2 7B?

Meta's 2023 dense 7B model. Still shows up in older llama.cpp scoreboards.

Stored parameters
7B
Active parameters
Dense
Q4 total (estimate)
9.0 GB
License
Llama 2 Community License

Model card. Meta lists Llama 2 7B as a 7B model.

Compatibility

HardwareQ4Q8FP16
M2 Ultra 192GB
Apple · 192 GB unified
9.0 GB
Fits
estimate · KV at 8k
Measured · 125 tok/s median
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Fits
estimate · KV at 8k
No measured run
RTX 4090
NVIDIA · 24 GB VRAM
9.0 GB
Fits
estimate · KV at 8k
Measured · 189 tok/s median
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Fits
estimate · KV at 8k
No measured run
M3 Ultra 512GB
Apple · 512 GB unified
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Fits
estimate · KV at 8k
No measured run
2× NVIDIA GB10
NVIDIA · 256 GB unified
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Fits
estimate · KV at 8k
No measured run
M3 Ultra 256GB
Apple · 256 GB unified
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Fits
estimate · KV at 8k
No measured run
M5 Ultra 256GB
Apple · 256 GB unified
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Fits
estimate · KV at 8k
No measured run
NVIDIA GB10
NVIDIA · 128 GB unified
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Fits
estimate · KV at 8k
No measured run
Ryzen AI Max+ 395 128GB
AMD · 128 GB unified
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Fits
estimate · KV at 8k
No measured run
M1 Ultra 128GB
Apple · 128 GB unified
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Fits
estimate · KV at 8k
No measured run
M5 Max 128GB
Apple · 128 GB unified
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Fits
estimate · KV at 8k
No measured run
4× RTX 3090
NVIDIA · 96 GB VRAM
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Fits
estimate · KV at 8k
No measured run
RTX Pro 6000 Blackwell
NVIDIA · 96 GB VRAM
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Fits
estimate · KV at 8k
No measured run
M5 Ultra 96GB
Apple · 96 GB unified
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Fits
estimate · KV at 8k
No measured run
Ryzen AI Max+ 395 64GB
AMD · 64 GB unified
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Fits
estimate · KV at 8k
No measured run
2× RTX 3090
NVIDIA · 48 GB VRAM
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Fits
estimate · KV at 8k
No measured run
M4 Pro 48GB
Apple · 48 GB unified
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Fits
estimate · KV at 8k
No measured run
M4 Max 36GB
Apple · 36 GB unified
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Fits
estimate · KV at 8k
No measured run
M5 Max 36GB
Apple · 36 GB unified
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Fits
estimate · KV at 8k
No measured run
RTX 5090
NVIDIA · 32 GB VRAM
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Fits
estimate · KV at 8k
No measured run
Radeon AI Pro R9700
AMD · 32 GB VRAM
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Fits
estimate · KV at 8k
No measured run
Intel Arc Pro B70
Intel · 32 GB VRAM
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Fits
estimate · KV at 8k
No measured run
M4 32GB
Apple · 32 GB unified
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Fits
estimate · KV at 8k
No measured run
M6 32GB
Apple · 32 GB unified
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Fits
estimate · KV at 8k
No measured run
RTX 3090
NVIDIA · 24 GB VRAM
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Fits
estimate · KV at 8k
No measured run
TITAN RTX
NVIDIA · 24 GB VRAM
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Fits
estimate · KV at 8k
No measured run
Radeon RX 7900 XTX
AMD · 24 GB VRAM
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Fits
estimate · KV at 8k
No measured run
M4 Pro 24GB
Apple · 24 GB unified
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Tight
estimate · KV at 8k
No measured run
M3 Pro 18GB
Apple · 18 GB unified
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Over
estimate · KV at 8k
No measured run
RTX 4060 Ti 16GB
NVIDIA · 16 GB VRAM
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Over
estimate · KV at 8k
No measured run
RTX 5060 Ti 16GB
NVIDIA · 16 GB VRAM
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Over
estimate · KV at 8k
No measured run
RTX 5070 Ti
NVIDIA · 16 GB VRAM
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Fits
estimate · KV at 8k
No measured run
18.8 GB
Over
estimate · KV at 8k
No measured run
M4 16GB
Apple · 16 GB unified
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Tight
estimate · KV at 8k
No measured run
18.8 GB
Over
estimate · KV at 8k
No measured run
RTX 3060 12GB
NVIDIA · 12 GB VRAM
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Over
estimate · KV at 8k
No measured run
18.8 GB
Over
estimate · KV at 8k
No measured run
RTX 3080 Ti
NVIDIA · 12 GB VRAM
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Over
estimate · KV at 8k
No measured run
18.8 GB
Over
estimate · KV at 8k
No measured run
RTX 4070 Ti
NVIDIA · 12 GB VRAM
9.0 GB
Fits
estimate · KV at 8k
No measured run
12.2 GB
Over
estimate · KV at 8k
No measured run
18.8 GB
Over
estimate · KV at 8k
No measured run
RTX 2060 Super
NVIDIA · 8 GB VRAM
9.0 GB
Over
estimate · KV at 8k
No measured run
12.2 GB
Over
estimate · KV at 8k
No measured run
18.8 GB
Over
estimate · KV at 8k
No measured run

Measured runs

HardwareQuanttok/sRuntimeSource
RTX 4090Q4_0188.96llama.cpp CUDA (FA)Source · checked · prefill 14771
M2 Ultra 192GBQ4_0125.21llama.cpp Metal (FA, Aug 2026 build)Source · checked · prefill 1489