Token

The unit a model reads and writes - roughly a word fragment. Pricing, context limits and speed are all measured in tokens.

Models do not see characters or words. Text is split into tokens, which are common character sequences learned from the training data. In English a token averages about four characters, so a hundred words is roughly 130 to 150 tokens. Code, rare names and non-English text tokenise less efficiently, sometimes at more than one token per character.

This matters for three reasons: your context-window is measured in tokens, you are billed per token in and out, and tasks that depend on individual characters - counting letters, reversing a string, rhyming - are hard for a model that never sees the characters.

Where this comes up

Prompts, configs and tutorials in the library that touch this term.