Better Models: Worse Tools
Armin reports a counterintuitive bug encountered while developing Pi code editor: newer Claude models, including Opus 4.8 and Sonnet 5, invent extra fields in the nested edits[] array when calling Pi's custom edit tool, causing the tool call to be rejected. Older Claude models do not exhibit this behavior. Armin hypothesizes that Anthropic's reinforcement learning has specifically optimized newer models for Claude Code's built-in edit tool, inadvertently degrading performance on other tool schemas. The piece questions whether third-party coding harnesses must implement multiple edit tools per model family, and highlights the fundamental trade-off between specialized training and general tool compatibility.
Better Models: Worse Tools. Armin reports on a weird problem he ran into while hacking on Pi:
The short version is that newer Claude models sometimes call Pi’s edit tool with extra, invented fields in the nested edits[] array. And not Haiku or some small model: Opus 4.8. The edit itself is usually correct but the arguments do not match the schema as the model invents made-up keys and Pi thus rejects the tool call and asks to try again.
Better Models: Worse Tools。Armin 报告了一个他在捣鼓 Pi 时遇到的奇怪问题：
简而言之，较新的 Claude 模型有时会在嵌套的 edits[] 数组中用额外的、凭空捏造的字段调用 Pi 的编辑工具。而且不是 Haiku 或某个小模型：是 Opus 4.8。编辑内容本身通常正确，但参数不符合 schema，因为模型发明了虚假的键，Pi 因此拒绝该工具调用并要求重试。
That alone is not too surprising as models emit malformed tool calls sometimes. Particularly small ones. What surprised me is that this is getting worse with newer Anthropic models as both Opus 4.8 and Sonnet 5 show it but none of the older models. In other words, the SOTA models of the family are worse at this specific tool schema than their older siblings.
这本身并不太令人惊讶，因为模型有时会发出格式错误的工具调用，尤其是小模型。令我惊讶的是，这个问题在较新的 Anthropic 模型中愈演愈烈——Opus 4.8 和 Sonnet 5 都表现出此问题，而所有旧模型则没有。换句话说，该系列中的 SOTA 模型在这个特定工具 schema 上的表现比它们的旧版更差。
Armin theorizes that this is because more recent Anthropic models have been specifically trained (presumably via Reinforcement Learning) to better use the edit tools that are baked into Claude Code. This has the unfortunate effect that other coding harnesses, such as Pi, may find that their own custom edit tools are more likely to be used incorrectly.
Armin 推测，这是因为较新的 Anthropic 模型经过了专门训练（可能通过强化学习），以更好地使用内置于 Claude Code 中的编辑工具。这产生了不幸的副作用：其他编码框架（如 Pi）可能会发现它们自己的自定义编辑工具更有可能被错误使用。
Claude's edit tool uses search and replace. OpenAI's Codex uses an apply_patch mechanism instead, and OpenAI have talked in the past about how their models are trained to use that tool effectively.
Claude 的编辑工具使用搜索和替换。OpenAI 的 Codex 则使用 apply_patch 机制，OpenAI 过去曾谈到他们如何训练模型有效使用该工具。
Does this mean third-party coding harnesses like Pi should implement multiple edit tools just so they can use the one with the best performance for the underlying model the user has selected?
这是否意味着像 Pi 这样的第三方编码框架应该实现多个编辑工具，以便根据用户选择的基础模型使用性能最佳的那个？