AnythingLLM

Point it at your documents and get a private assistant over them

AnythingLLM is a self hosted application built around retrieval over your own material: drop in documents, and it handles the chunking, the embeddings and the vector store, then answers over them. It runs as a desktop app or a server with workspaces and users.

Free tierMITJavaScriptSelf hostable

It hides the parts nobody enjoys

Retrieval sounds simple and is mostly plumbing: chunk sizes, embedding models, a vector database, and the glue between them. AnythingLLM makes reasonable choices for all of it so you can find out whether retrieval helps your case before spending a week assembling one.

Workspaces are the useful organising idea. Each has its own documents and its own thread, so a set of contracts and a set of technical docs stay separate rather than blending into one confused context.

Where the ceiling is

Convenient defaults are still defaults. When answers are poor, the levers you want are the chunking strategy and the retrieval logic, and those are less exposed here than in a pipeline you assembled yourself.

The desktop application is the fastest way to evaluate it. Treat that as the trial and the server deployment as the real thing, because they are not equally suited to more than one person.

Reach for it when

you want to know whether retrieval helps before building one
the documents must stay on infrastructure you control
you need separate contexts for separate bodies of material

Watch out for

the retrieval internals are less exposed than a pipeline you built, which limits how far you can tune it
a hosted cloud offering is sold alongside the open source project
answer quality depends on the embedding and chat models you configure, so judge those separately

NEXT TO IT ON THE SHELF

AnythingLLM

It hides the parts nobody enjoys

Where the ceiling is

Reach for it when

Watch out for

Compare with these

Open Generative AI

Open WebUI

LibreChat

New entries, as they land