LM Studio
A desktop app for downloading, running and comparing local models without a terminal. Best tool for deciding which quantisation is good enough before you commit to it.
Tested on Llama 3.2 8B ·
APP · Free for personal use
LM Studio answers one question: which of these models fits on my machine, and is it still useful at that size? You browse models, see the memory estimate before you download, and load two side by side.
What it does well. The memory estimate before the download saves the most time. Comparing quantisations is easy enough that you will do it, and the answer is often that a smaller quant is fine for your task. It can also serve an OpenAI-compatible endpoint, so it doubles as a local server.
What it does badly. It is a desktop app, so a headless box is out. The catalogue mixes well-kept conversions with abandoned ones and the interface does not tell you which is which. Updates sometimes move the folder where models are stored.
What I would use instead. Ollama, once you know the model and quantisation you want and prefer to script it.
Notes from the author
Run a 4-bit and an 8-bit quant of the same model on your own prompts before you assume you need the bigger one. For extraction and classification the difference is usually invisible, and the smaller file leaves memory free for a longer context.
For information only, not professional advice. You use it at your own risk and we are not liable for any loss or damage. See the disclaimer.
Tagged