How does tokenization impact model input constraints and prompt design in practice?

• A. Tokenization determines the number of tokens in a prompt, affecting the maximum length available for the answer and the cost of processing.
• B. Tokenization has no effect on input length or cost.
• C. Tokenization only affects training speed.
• D. Tokenization is irrelevant to prompt design.

Answer: (A)

Tokenization is the process of converting text into tokens that the model processes. The number of tokens a prompt becomes determines how much of the model’s fixed context window is used and, therefore, how long the model’s response can be. Since many APIs charge per token and the model can only attend to a limited total number of tokens, longer prompts leave less room for the answer and increase cost. Tokenization also matters because different tokenizers break text into tokens in different ways; the same sentence can produce more or fewer tokens depending on the model. That means what looks short in characters can still consume a lot of the token budget, affecting both input length and price. When designing prompts, you must consider this token budget, aim for concise yet clear prompts, and structure content to minimize token waste while preserving the necessary context.