Choosing a model for the job

Benchmark tables age within weeks and rarely predict how a model behaves on your work. The shape of the task is a better guide, and it changes far more slowly.

Long input, close instruction-following

Code review, editing a long document, anything with a list of constraints to honour. Claude handles this well and reacts strongly to clearly marked sections, so wrapping input in delimiters helps more than rewording the instruction. See the claude page for what changes when you move a prompt there.

Structured output and tool use

Extraction into JSON, function calling, agent loops. GPT models are strongest here, and their custom instructions and project files keep your rules out of every message.

Very large or mixed input

Whole PDF sets, video, hours of audio. Gemini takes inputs the others cannot, which changes the approach: hand over the whole thing and ask for a targeted extraction instead of chunking.

Private or offline data

If the data cannot leave your machine, the decision is made for you. Llama or Mistral through Ollama or LM Studio. Expect to simplify: a quantised model follows a four-part instruction less reliably, so split tasks and keep the output format simple.

Hard multi-step reasoning

Maths, tricky logic, a bug that resists the obvious explanation. A reasoning-model earns its latency here and nowhere else. On extraction and formatting it is slower for no gain.

Cost

For classification and short extraction at volume, a small fast model at low temperature is usually as accurate as a large one and an order of magnitude cheaper. Test the cheap one first; upgrade only where it measurably fails.

How to actually decide

Take three real tasks from your week. Run each on two candidates with the same prompt at temperature 0. Compare the outputs yourself.

That afternoon tells you more than any leaderboard, because it is measured on your work. Redo it when a major version ships - roughly twice a year is enough.