别只看窗口大小：token 上限与 lost in the middle 才是关键
上下文窗口由输入 token（system prompt 与 user prompt）和输出 token（模型回复）共同构成，模型能看到的 token 总数存在硬性上限。对话越长，越容易在请求时或生成中途触发 API 错误，而且模型无法自行绕过这一限制。更关键的问题是：上下文窗口越大，lost in the middle 现象越明显，模型对中间部分信息的注意力越低，甚至可能无法从自己的上下文中取回所需内容。因此评估模型时不能只看窗口大小，即使支持超长上下文，使用更少的 token 往往能换来更稳定的结果。本文是 LLM 基础概念的短篇讲解，适合刚开始接触 LLM 或正在设计 Agent 上下文的工程师快速建立全局认识。
The context window is made up of input and output tokens. The input tokens might include a system prompt and a user prompt (the message from the user). And then the output tokens is whatever the assistant comes back with:
The context window is the input and output tokens combined.
上下文窗口由输入 token 和输出 token 构成。输入 token 可能包括系统提示（system prompt）和用户提示（user prompt，即用户发来的消息）；输出 token 则是助手给出的回应：
上下文窗口就是输入和输出 token 的总和。
As the conversation gets longer, as more and more messages get put into the conversation, the number of tokens used grows.
And as you can imagine, this can't go on forever. Every model has a hard-coded limit for the number of tokens it can see at any one time.
对话越来越长，放进对话里的消息越来越多，用掉的 token 数量也随之增长。
可想而知，这不可能无限持续下去。每个模型都有一个硬编码（hard-coded）的上限，规定它同一时刻最多能看到的 token 数量。
Context Window Limits
If we imagine a super long conversation, eventually we will hit a limit. And if you try to query the LLM with this super long conversation and you hit the limit, then you will probably get some kind of error back from the API.
You might even hit the limit during the generation of a message. So this message might start outside the context window limit, but as it goes or it carries on until it hits the context window limit.
The model itself isn't quite smart enough to work around its own context window limits. And so you will hit issues like this sometimes.
上下文窗口上限
想象一段超长对话，我们最终会触达上限。如果你试图把这段超长对话发给 LLM，一旦撞上上限，API 大概率会返回某种错误。
你可能在消息生成过程中就触达上限。也就是说，这条消息开始时可能还在上下文窗口限制之内，但一边生成一边推进，直到触及上下文窗口上限。
模型本身并不够聪明，无法自行绕开自己的上下文窗口上限。所以你有时候就会遇到这类问题。
But the biggest issue with context windows is that the bigger they get, the more "lost in the middle" issues you get.
If we imagine a huge conversation, where these rings are the individual messages, the messages at the start of the history have quite a big impact on the output, and the ones at the end do too, but the stuff in the middle the LLM pays a bit less attention to.
This is a well-known phenomenon and it's much more pronounced the larger the context window gets.
但上下文窗口最大的问题是：窗口越大，“lost in the middle”（迷失在中间）的情况就越严重。
想象一段很长的对话，图中这些圆环就是一条条独立的消息。历史记录开头的消息对输出影响很大，结尾的消息同样影响很大，但中间的部分，LLM 给到的注意力会少一些。
这是一个广为人知的现象，而且上下文窗口越大，这种现象就越明显。
So when you're assessing a model, you shouldn't just think, "wow, this context window limit's huge, that's incredible, I can put so much stuff in there."
That model will still probably have lost-in-the-middle issues. It may even have trouble retrieving information from its own context window.
And so even if that model supports a big context window, you'll definitely still get better results from using fewer tokens in the context.
所以在评估一个模型时，不要只想着：“哇，这个上下文窗口上限好大，太厉害了，我可以往里塞好多东西。”
那个模型仍然大概率会出现中间迷失问题；它甚至可能连从自己的上下文窗口里检索信息都有困难。
所以，即便那个模型支持很大的上下文窗口，在上下文里使用更少的 token，效果仍然会更好。