Esc

Token

The unit a model reads and writes, roughly a word fragment. Price, context limits and speed are all counted in tokens.

Models never see letters or whole words. Text gets cut into tokens, common character sequences picked up from the training data. English averages about four characters per token, so a hundred words comes to somewhere between 130 and 150 tokens. Code, rare names and non-English text split less efficiently, sometimes into more than one token per character.

You care for three reasons. The context-window is counted in tokens. You pay per token in and out. And any task that depends on single characters, such as counting letters, reversing a string or finding a rhyme, is hard for a model that never sees them.

Where this comes up

Prompts, configs and tutorials in the library that touch this term.