Quantisation
Storing a model with lower-precision numbers so it needs less memory and runs on ordinary hardware.
Model weights usually start as 16-bit floats. Quantisation stores them in 8, 5 or 4 bits and cuts memory roughly in proportion. A model that wants a data-centre GPU at full precision can run on a laptop at 4-bit.
Quality slips gradually. At 8-bit you will struggle to see a difference. At 4-bit most tasks still work, though long instruction lists and exact formatting go first, and those are what prompts lean on. Below 4-bit the loss is obvious.
That is the main reason a prompt that runs fine on a hosted model needs simplifying for a local one. The Llama platform page lists what to change.
Tagged
Where this comes up
Prompts, configs and tutorials in the library that touch this term.
- ConfigLlama
Modelfile for a consistent local persona
A Modelfile that fixes the system prompt and sampling parameters of a local model, so the same behaviour holds in every session and you stop retyping…
- Local setupLlama
Running your first local model
Ollama on an ordinary laptop, from install to a model answering in about ten minutes. It also shows how to work out which size fits before you…
Tested on Llama 3.2 8B · May 2026
- CheatsheetLlama
Ollama commands worth knowing
The part of the CLI you will use, plus two flags that fix the problems everyone hits.
Tested on Llama 3.3 70B · May 2026
LM Studio
A desktop app for downloading, running and comparing local models without a terminal. Best tool for deciding which quantisation is good enough before…
Tested on Llama 3.2 8B · May 2026
Ollama
The easiest way to get a local model answering on your own machine. One command installs it, one pulls a model, and an API appears on localhost that…
Tested on Llama 3.3 70B · May 2026
- ConfigLlamaMistral
Ollama Modelfile for a local code assistant
A Modelfile with a system prompt and parameters tuned for short, correct code answers on a local model.