30-SECOND SUMMARY
What to take away
- Bad extraction and retrieval cannot be repaired reliably by a larger generator.
- Use the same embedding model for indexing and querying.
- Evaluate retrieval and answer generation separately with questions whose source locations are known.
From document to grounded answer
Retrieval and generation fail in different ways.
- 01SPLIT
Meaningful chunks
- 02EMBED
One embedding model
- 03SEARCH
Retrieve evidence
- 04ANSWER
Cite the source
The RAG pipeline
Extract text, split it into meaningful chunks, create embeddings, retrieve the closest chunks for a question, and place those chunks in the generation prompt.
Store file, page, heading, and chunk identifiers so every answer can link back to evidence.
Fix extraction before retrieval
PDF reading order, tables, headers, and scanned pages often produce broken text. Compare extracted samples with the original before indexing thousands of pages.
Remove duplicates and obsolete versions, and keep documents with different access permissions in separate indexes.
Chunk by meaning
Very short chunks lose context; very long chunks add noise. Prefer headings and paragraph boundaries, then use overlap only where continuity is genuinely needed.
Each known answer should be traceable to a specific chunk ID.
Create and search embeddings
Ollama exposes `/api/embed` for local embeddings. Use the same embedding model for documents and questions and compare vectors with an appropriate similarity measure.
Inspect top results manually before adding filters, hybrid search, or reranking.
curl http://localhost:11434/api/embed -d '{
"model": "embeddinggemma",
"input": ["Refunds are available within seven days."]
}'Evaluate retrieval and generation
Create questions for single facts, multiple sources, similar documents, and answers that do not exist. First check whether the correct chunk was retrieved; then check whether the model used it accurately.
When deleting a document, remove source text, chunks, embeddings, caches, and related logs, then confirm that it can no longer be retrieved.
Frequently asked questions
Is RAG the same as training?
No. Typical RAG leaves model weights unchanged and supplies retrieved context at request time.
Must the embedding and chat model be the same?
No, but document indexing and query embedding should use the same embedding model.
Does adding more documents always help?
No. Duplicates, obsolete versions, and mixed permissions can reduce quality and create security problems.
Primary sources
Check the original documentation for version-specific details.
Ollama Embeddings Ollama API Introduction