LM Studio
A desktop app for downloading, running and comparing local models without a terminal. Best tool for deciding which quantisation is good enough before you commit to it.
Tested on Llama 3.2 8B ·
APP · Free for personal use
LM Studio is the answer to "which of these models actually fits on my machine and is it still useful at that size". You browse models, see the memory estimate before downloading, and load two side by side.
What it does well. The memory estimate before download is the feature that saves the most time. Quantisation comparison is easy enough that you will actually do it, and the answer is often that a smaller quant is fine for your task. It can also serve an OpenAI-compatible endpoint, so it doubles as a local server.
What it does badly. It is a desktop app, so it is not what you want on a headless box. The model catalogue mixes well-maintained conversions with abandoned ones and the interface does not distinguish them clearly. Updates occasionally change where models are stored.
What I would use instead. Ollama, once you know which model and quantisation you want and would rather script it.
Notes from the author
Compare a 4-bit and an 8-bit quant of the same model on your own prompts before assuming you need the larger one. On extraction and classification the difference is usually invisible, and the smaller one leaves memory for a longer context.
Tagged