30-SECOND SUMMARY
What to take away
- Context is the token window available to the model.
- Longer context increases KV-cache memory.
- Measure the smallest sufficient value for the workload.
How context consumes memory
Budget tokens before raising limits.
- 01PROMPT
Instructions and text
- 02TOKENS
Shared budget
- 03KV CACHE
Length and concurrency
- 04LIMIT
Smallest sufficient value
Budget tokens
System instructions, history, retrieved text, and the answer share one context budget. Characters do not map to tokens uniformly across languages.
Account for KV cache
The runtime stores prior-token attention state in a KV cache. Longer context and more concurrent requests consume more memory even with the same model file.
Inspect Ollama
Ollama documents VRAM-based defaults and warns that larger context requires more memory. Use `ollama ps` to inspect allocation and offloading.
OLLAMA_CONTEXT_LENGTH=8192 ollama serve
ollama psIncrease only as needed
Measure representative documents plus instructions and answer headroom. Consider retrieval or hierarchical summaries instead of inserting everything.
Frequently asked questions
Does more context always improve answers?
No. It costs memory and latency, and models may not use it effectively.
Does it use RAM or VRAM?
That depends on the runtime and offloading; inspect the running process.
What if long chats slow down?
Compare a new chat and a smaller context with the same prompt.
Primary sources
Check the original documentation for version-specific details.
Ollama Context Length Ollama FAQ