arosplatforms™AI consultancy

AI

ar
← AI Glossary
Core concepts

Token

The basic chunk of text a language model reads and writes, often a word, part of a word, or punctuation mark.

A token is the unit a language model actually processes. Text is broken into tokens before the model sees it, and a token can be a whole word, a piece of a word, or a punctuation mark. As a rough guide, one token is about four characters of English, so a hundred words is roughly a hundred and thirty tokens.

Tokens matter because they drive both cost and capacity. Most AI providers bill per token of input and output, and every model has a limit on how many tokens it can handle at once. Understanding token counts is how teams predict spend and avoid hitting size limits mid-task.

At arosplatforms we estimate token usage early when sizing a project, because it directly shapes cost per request and which model is the right fit. Designing prompts and retrieval to be token-efficient is one of the simplest ways we keep client AI costs predictable.

Have a use for this in your business?

Book a free consultation and we'll show you what's feasible and how we'd ship it.