Context Windows Are Not About Size: Tokens, Limits, and Lost in the Middle
The context window is the combined total of input tokens (system prompt, user prompt) and output tokens (the assistant's reply). Every model has a hard-coded token limit, so long conversations eventually produce API errors—sometimes mid-generation. Models can't work around their own limits, and a larger window actually worsens the well-known "lost in the middle" effect: the beginning and end of the conversation get more attention while the middle fades, making the model less able to retrieve its own context. Even with a huge window, you'll get better results from using fewer tokens. This is a concise explainer for engineers getting started with LLMs or designing agent context.