Choosing a model for the job

Benchmark tables go stale in weeks and rarely tell you how a model behaves on your work. The shape of the task is a better guide, and it changes much more slowly.

Long input, close instruction-following

Code review, editing a long document, anything with a list of constraints to honour. Claude does well here and reacts strongly to marked sections, so wrapping the input in delimiters helps more than rewording the instruction. The claude page lists what changes when you move a prompt there.

Structured output and tool use

Extraction into JSON, function calling, agent loops. GPT models are strongest here, and custom instructions and project files keep your rules out of every message.

Very large or mixed input

Whole PDF sets, video, hours of audio. Gemini takes inputs the others cannot, and that changes the method: hand over the lot and ask for one targeted extraction, no chunking.

Private or offline data

If the data cannot leave your machine, the choice is made. Use Llama or Mistral through Ollama or LM Studio and expect to simplify. A quantised model follows a four-part instruction less reliably, so split the task and keep the output format plain.

Hard multi-step reasoning

Maths, tricky logic, a bug that resists the obvious explanation. A reasoning-model earns its delay here and nowhere else. On extraction and formatting it is just slower.

Cost

For classification and short extraction at volume, a small fast model at low temperature is usually as accurate as a large one and about ten times cheaper. Test the cheap one first and move up only where it visibly fails.

How to decide

Take three real tasks from your week and run each on two candidates with the same prompt at temperature 0. Compare the outputs yourself.

That afternoon tells you more than a leaderboard, since it is measured on your work. Repeat it when a major version ships, about twice a year.