30-SECOND SUMMARY
What to take away
- Measure load, first output, and sustained generation separately.
- Repeat with fixed prompt, context, and output length.
- Record worst cases, memory, failures, and quality.
Split perceived speed
Separate cold and warm runs.
- 01LOAD
Model loading
- 02TTFT
First output
- 03TOKENS
Sustained generation
- 04QUALITY
Task success
Separate three phases
Cold start includes model loading. Time to first token describes interactive waiting, while tokens per second describes continued generation.
Fix the conditions
Record model, quantization, runtime, context, prompt, output cap, power mode, and background workload. Separate cold and warm runs.
Read API metrics
Ollama generate and chat responses expose total, load, prompt-evaluation, and generation fields. Check units and the final streaming object.
curl http://localhost:11434/api/generate -d '{"model":"gemma3:4b","prompt":"Explain local AI","stream":false}'Pair speed with success
A fast wrong answer is not a better model. Compare performance only among candidates that pass the task-quality threshold.
| Phase | Meaning |
|---|---|
| Load | Move model into memory |
| TTFT | Request to first output |
| Generation | Output after first token |
| Total | End-to-end task time |
Frequently asked questions
Is tokens per second enough?
No. Interactive and batch work value different phases.
Can I measure once?
Repeat and record median plus worst case.
Can models with different outputs be compared?
Use output limits and include quality scores.
Primary sources
Check the original documentation for version-specific details.
Ollama Generate API Ollama Chat API