Quantisation

Storing a model with lower-precision numbers so it fits in less memory and runs on ordinary hardware.

Model weights are normally 16-bit floats. Quantisation stores them in 8, 5 or 4 bits, cutting memory roughly in proportion. A model that needs a data-centre GPU at full precision can run on a laptop at 4-bit.

Quality degrades gradually rather than falling off a cliff. At 8-bit the difference is hard to detect; at 4-bit most tasks still work but long instruction lists and precise formatting suffer first, which is exactly what prompts depend on. Below 4-bit the loss is obvious.

This is the main reason a prompt that works on a hosted model needs simplifying for a local one. See the Llama platform page for what to change.

Where this comes up

Prompts, configs and tutorials in the library that touch this term.