Ollama
Run open language models locally behind a simple API
Ollama downloads and runs open weight language models on your own machine, with one command per model and a local API that many apps support. It is MIT licensed, uses your GPU when it can, and is the usual engine behind local AI front ends.
Best for when
- you want to run language models on your own GPU
- you work with private data that should not leave your machine
- you want a local API for chat interfaces, editors or agents
Watch out for
- model size is capped by your GPU memory, and larger models get slow
- local models trail the best hosted models on hard tasks
- the API has no authentication, so never expose it to the internet
Install
Install on Linux
curl -fsSL https://ollama.com/install.sh | shLocal models, minus the setup pain
Running a model used to mean picking a format, a quantisation and a runtime by hand. Ollama turns it into pulling a model by name and talking to it from the terminal or over its HTTP API, which chat interfaces, editors and agents already know how to call.
Everything stays on your machine: prompts, documents and answers. That makes it a good fit for private data, offline work, and experiments where per token pricing would add up.
Your hardware is the limit
The model has to fit in memory. With an 8 GB GPU you are in small and medium model territory, and anything larger spills to system RAM and slows down sharply. Models are several gigabytes each on disk, and they add up quickly.
Local models are not as capable as the best hosted ones, so match the task to the model. The API has no authentication by default, so keep it bound to localhost or your private network. The company also sells a paid cloud for larger models.
