local models
Running models on your own hardware.
14 tagged items
- GuideUniversal
Ways to use AI without paying
The free ways to use AI, where each one runs out, and a simple test for whether free is enough for your task.
- ConfigLlama
Modelfile for a consistent local persona
A Modelfile that fixes the system prompt and sampling parameters of a local model, so the same behaviour holds in every session and you stop retyping…
- Local setupLlama
Running your first local model
Ollama on an ordinary laptop, from install to a model answering in about ten minutes. It also shows how to work out which size fits before you…
Tested on Llama 3.2 8B · May 2026
Groq Cloud
Hosted inference on custom LPU hardware, built for speed. Open-weight models answer at a token rate that makes streaming feel close to instant.
Tested on Llama 3.3 70B · Jun 2026
Together AI
Hosted inference for open-weight models such as Llama, Mixtral and Qwen, plus fine-tuning, at a lower price than most proprietary-model APIs.
Tested on Llama 3.3 70B · Jun 2026
AnythingLLM
A desktop app for chatting with your own documents against a local or hosted model. The retrieval pipeline is built in, so you do not assemble it…
Tested on Llama 3.3 70B · Jun 2026
- CheatsheetLlama
Ollama commands worth knowing
The part of the CLI you will use, plus two flags that fix the problems everyone hits.
Tested on Llama 3.3 70B · May 2026
LM Studio
A desktop app for downloading, running and comparing local models without a terminal. Best tool for deciding which quantisation is good enough before…
Tested on Llama 3.2 8B · May 2026
Ollama
The easiest way to get a local model answering on your own machine. One command installs it, one pulls a model, and an API appears on localhost that…
Tested on Llama 3.3 70B · May 2026
- ConfigLlamaMistral
Ollama Modelfile for a local code assistant
A Modelfile with a system prompt and parameters tuned for short, correct code answers on a local model.
- GuideLlamaMistral
Running a model on your own machine
What local models are genuinely good for, what hardware you need, and what to change in prompts that came from a hosted model.
- GuideUniversal
Choosing a model for the job
A practical comparison of what to reach for when, based on the shape of the task rather than benchmark scores.
Related terms
Inference
Running a trained model to get output. It happens every time you send a prompt.
Quantisation
Storing a model with lower-precision numbers so it needs less memory and runs on ordinary hardware.