Better Models: Worse Tools
Pi author discovers that Anthropic's Opus 4.8 and Sonnet 5 inject spurious keys (requireUnique, oldText2, cost, etc.) into the edits[] array of Pi's edit tool, while older models do not. The failure is context-dependent and reproducible in agentic sessions. The post dissects Anthropic's tool calling internals: ANTLM markers, JSON-serialized nested arrays, and Claude Code's extremely forgiving harness that silently filters unknown keys and retries malformed calls. Author hypothesizes that RL post-training over Claude Code's flat old/new_string schema creates a strong prior, making newer models worse at following non-canonical tool schemas. Strict tool invocation fixes the issue, but Anthropic's complexity limits prevent Claude Code from using it. Key takeaway: tool schemas are not distribution-neutral; any harness must inherit Claude Code's quirks.