Local AI Privacy and Security Checklist

VERIFICATIONBased on official Ollama data-handling guidance, llama.cpp security policy, and OWASP GenAI security resources.

30-SECOND SUMMARY

What to take away

‘Local’ describes inference location, not the entire data lifecycle.
Separate local models from cloud models and web tools.
Treat model files, logs, RAG indexes, backups, and open ports as part of the attack surface.

DATA PATH 01

Trace every copy of a document

Local inference does not describe the full lifecycle.

01
SOURCE
Original file
02
INDEX
Chunks and vectors
03
INFERENCE
Prompt and response
04
STORAGE
Logs and backups

Use it this way Define access, retention, and deletion at each stage.

SECTION 01

Draw the complete data path

Track the original document, extracted text, chunks, embeddings, model input, generated response, logs, backups, and deletion process. Each stage can create another copy.

Record storage location, authorized users, retention period, and deletion method for every stage.

SECTION 02

Separate local and cloud features

Ollama distinguishes local processing from cloud models and web search. A model name ending in `:cloud` is not an on-device workload.

Disable unneeded cloud features and verify behavior with the network disconnected before adding private documents.

SECTION 03

Treat model files as untrusted input

Model formats are complex binary inputs consumed by parsers. Use trusted publishers, verify hashes when provided, update runtimes, and test unfamiliar files in an isolated environment.

A downloadable model is not automatically safe or licensed for every business use.

SECTION 04

Do not expose an unauthenticated server

A service bound only to localhost has a smaller network surface than one bound to all interfaces. Tunnels and reverse proxies can silently turn a personal model server into an internet service.

External or multi-user access requires authentication, TLS, network controls, rate limits, monitoring, and tenant isolation.

SECTION 05

Check logs, sync, and deletion

Debug logs can contain prompts, file paths, or responses. Cloud backup and desktop sync tools may copy local artifacts to remote storage.

Deleting the source document is not enough when chunks, embeddings, caches, and chat history remain. Test deletion across the entire pipeline.

FAQ

Frequently asked questions

Is an offline computer completely safe?

No. Offline operation reduces external transfer but does not remove malicious files, local access, logs, backups, or prompt-injection risks.

Can I use company documents?

Only after checking company policy, data classification, model licensing, access controls, retention, and deletion requirements.

Does RAG remove the need to protect the source?

No. The index and embeddings become additional sensitive assets that require access control and deletion.

Primary sources

Check the original documentation for version-specific details.

Ollama FAQ

llama.cpp Security Policy

OWASP GenAI Security Project