How Much RAM and VRAM Do Local LLMs Need?

VERIFICATIONWeight-size calculations use published GGUF bits-per-weight values; they are theoretical lower bounds, not measured RAM requirements.

30-SECOND SUMMARY

What to take away

Estimate weight size with parameters × bits per weight ÷ 8.
Leave headroom for KV cache, the runtime, and the operating system.
A smaller model that stays fully accelerated can feel better than a larger model that constantly spills to CPU memory.

MEMORY 01

What consumes memory

Model file size is only the first layer.

01
WEIGHTS
Quantized parameters
02
CONTEXT
KV cache
03
RUNTIME
Buffers and backend
04
SYSTEM
OS and applications

Use it this way Plan with headroom instead of filling the last available gigabyte.

SECTION 01

RAM, VRAM, and unified memory

System RAM is used by the operating system, CPU inference, and application state. Dedicated GPU VRAM stores data used directly by the GPU. Apple Silicon uses a unified memory pool shared by CPU and GPU.

These architectures are not directly interchangeable. Always test with the runtime and model you intend to use.

SECTION 02

Estimate model-weight size

A rough lower bound is parameter count multiplied by average bits per weight, divided by eight. An 8B model at roughly 4.5 bits per weight is about 4.5GB for weights alone.

Metadata, alignment, runtime buffers, and architecture details can change the actual file and memory footprint.

weight GB ≈ parameters (billions) × bits per weight ÷ 8

SECTION 03

Context consumes additional memory

The KV cache grows with context length, model architecture, precision, and concurrent requests. Doubling context can materially increase memory even though the model file stays unchanged.

Use the smallest context that fits the real task and begin with one request at a time.

SECTION 04

Practical starting ranges

8GB systems are best treated as experiment machines for very small models. 16GB is a practical entry point for many 3B–8B Q4 models. 32GB opens more room for 8B–14B models and document workflows.

These are planning ranges, not universal minimums. CPU features and runtime support may prevent a model from running even when memory appears sufficient.

Memory	Practical starting point	Typical constraint
8GB	1B–3B Q4	Short context, close other apps
16GB	3B–8B Q4	Limited large-model headroom
32GB	8B–14B Q4 candidates	Context and speed still matter
64GB+	14B–32B Q4 candidates	Validate throughput and power

SECTION 05

Measure instead of guessing

Run the same prompt with one model at a time. Record peak memory, time to first output, sustained generation, and whether the model remains fully accelerated.

Stop when quality meets the task. Buying hardware for a larger model that does not improve your workflow is wasted capacity.

FAQ

Frequently asked questions

Is file size equal to required RAM?

No. The runtime, context cache, operating system, and other applications require additional memory.

Is more VRAM always better?

Capacity matters, but bandwidth, software support, model architecture, and workload also affect performance.

Can a model larger than VRAM still run?

Some runtimes can split work across GPU and system memory, but performance may fall substantially.

Primary sources

Check the original documentation for version-specific details.

Hugging Face GGUF

Ollama Context Length

llama.cpp Repository