AnythingLLM
A desktop app for chatting with your own documents against a local or hosted model. The retrieval pipeline is built in, so you do not assemble it yourself.
Tested on Llama 3.3 70B ·
APP · Free, open source; optional paid cloud hosting
AnythingLLM puts document ingestion, chunking, embedding and retrieval behind an ordinary chat window. Asking questions about a folder of PDFs or notes needs no vector database of your own. Pair it with a local Ollama model to work fully offline, or with a hosted API when you want stronger answers.
Retrieval quality depends a lot on how your documents are organised. A folder of clean, well-titled files does far better than a dump of scanned PDFs with no structure, as in any retrieval system.
Notes from the author
Begin with a small, tidy document set and not everything you own. Bad retrieval on fifty files is easy to diagnose. On ten thousand mixed files it is not.
For information only, not professional advice. You use it at your own risk and we are not liable for any loss or damage. See the disclaimer.
Tagged