标签 · Cost Optimization — Glean

Recent picks

24picks · chronological

09-09

Forgotten Agent Ran 1,555 Sessions a Day and Ate My Claude Quota

The author's Claude Max quota drained in 10 minutes, and he nearly blamed a faulty meter. Digging through Claude Code's local logs, he found a forgotten background agent from a YC batch project spawning 1,555 short-lived sessions a day, with 91% of usage generated by machines rather than by him. The post explains why the limits feel broken: the 5-hour cap is a rolling window that background sessions can already fill, and every fresh session pays a cache-warmup cost roughly 12 times higher than re-reading from cache. The author turned his two-day investigation into tare, an open-source Claude Code skill that deduplicates log entries (naive counts overstate tokens by about 86%) and answers with a cause instead of a raw spreadsheet. It runs entirely locally, makes zero network calls, and can produce a scrubbed HTML usage report that can be shared without leaking prompts or file paths. Useful for anyone whose Claude Code quota keeps disappearing and for teams auditing agent-side spending.

www.kelviq.com · 8 min · Agent Engineering · Ai Tooling · Claude Code

08-17

Harness Swap Lifts Same Model from 46.7% to 66.7% Success

In a Composio benchmark, DeepSeek V4 Flash running across 8 agent harnesses passed 20 of 30 hard tasks with Pi (66.7%), while Claude Code, Codex, and Deep Agents each passed 16. Pi's cost per successful task was $0.028, about 1/7 of Claude Code's $0.195. Community numbers show Pi and DeepSeek hitting 99.93% cache hits, processing nearly 1B input tokens for just $2.65 instead of an estimated $132. The article explains DeepSeek's prefix cache: matches must start from the first token, so harnesses must keep the prompt head stable. Reasonix and pi-deepseek-cache demonstrate concrete tactics: frozen environment summaries at startup, append-only context, separate sessions for executor and planner, and hash-cached deterministic summaries, cutting input token prices by 98-99%. Useful for engineers choosing agent harnesses or optimizing LLM API spend.

www.infoq.cn · 6 min · Agent Engineering · Agent Infrastructure · Cost Optimization

08-16

Maximizing the value of your Claude Code sessions

Claude Code bills tokens in two phases: prefill reads the whole request, decode generates output one token at a time, which is why output costs about 5x input. Prompt caching is automatic: shared prefixes are served at 0.1x, but switching /model, /effort, or fast mode, or running /compact, invalidates the cache and forces a full-price re-prefill. Session cost is determined by how many tokens enter the context, how many turns they stay, and how many contexts run in parallel. The guide offers concrete tactics: use @-mentions to avoid Read calls, put quiet flags for daily commands into CLAUDE.md, spill outputs over 30,000 characters to a file, use /rewind instead of /compact to trim dead ends, and offload noisy jobs to subagents (optionally pinned to haiku). Cache expiry is one hour on subscription and five minutes on API keys, extendable via ENABLE_PROMPT_CACHING_1H=1. Practical for heavy Claude Code users.

claude.com · 13 min · Claude Code · Context Engineering · Cost Optimization

08-16

Graph Engineering: from 1 prompt to 100 agents in one system

This article argues that the workflow itself is the engineering target: alongside prompt, context, harness, and loop engineering, there is a fifth layer the author calls graph engineering. The core observation is that default linear agent flows confuse sequence with dependency: most arrows do not read an upstream result, so cutting them is what enables real parallelism. A hundred agents, in this view, are not a hundred roles but one role instantiated a hundred times with its own slice of the problem. The eight-step method covers node contracts, four topologies (chain, fan, router, controlled cycle), when joins are worth waiting for, separating probabilistic classification from deterministic routing, placing independent verifier nodes, and keeping durable state. It explicitly warns that parallel breadth is expensive, citing Anthropic's multi-agent research system consuming roughly 15x the tokens of a normal chat. Useful for engineers moving from a single prompt to a multi-agent production system, though there is no runnable code.

x.com · 10 min · Agent Architecture · Agents · Context Engineering

08-13

Grok 4.6 and DeepSeek V4 Pro: near-Fable power, far lower price

A heavy Agent user shares a first-hand model-selection update after DeepSeek V4 Pro and Grok 4.6 launched within hours of each other. Key data: DeepSeek V4 Pro costs $0.87 per million output tokens, roughly 1/57 the price of Claude Fable 5, scores 87.9 on Terminal Bench 2.1 versus Fable 5's 88.0, and lifts DeepSWE from 12.8 to 62.7. Grok 4.6, priced at $2/$6 per million tokens, finished most development tasks in under 20 minutes; four hours of heavy use consumed only 2% of the author's weekly SuperGrok Heavy quota. The post also admits the impossible triangle is not fully broken: DeepSeek still lacks multimodal support and is not fast, while Grok lags Fable 5 on complex engineering and agent work. Useful for engineers comparing model cost, speed, and agent capability.

x.com · 6 min · Agent Engineering · AI Engineering · Cost Optimization

08-10

5 Questions to Ask Before Choosing an LLM

Choosing an LLM is not a one-time decision—it must be revisited as models and your app evolve. This guide breaks the choice into five questions: open vs. closed source, cost, latency, performance, and context window. Open models require self-hosting or an API provider like Hugging Face or Groq; closed models are hosted by vendors and priced per token. Latency should be measured with TTFT and TPOT. Benchmarks like Chatbot Arena and Open LLM Leaderboard are useful early indicators, but prone to overfitting—the only real test is running your own evals within your application. Reasoning models such as o1 excel at planning-heavy tasks but cost more and respond slower. Context windows count both input and completion tokens, which is why RAG chunking exists. A practical primer for developers starting model selection.

www.aihero.dev · 8 min · AI Engineering · Context Engineering · Cost Optimization

08-08

Cloudflare Computer: How to Cut AI Agent Sandboxing Costs by 80%

The default way to sandbox AI agents is to keep a full Linux container alive for every agent. Cloudflare Computer proposes a split: the Workspace Durable Object (with a SQLite VFS) owns authoritative project state; ordinary reads, searches and edits run in a Worker isolate via workspace.fs and just-bash; real Linux operations like npm install and build start a container on demand, and a post-command pull synchronizes changes back. Using a small Vite site as the test case, the author shows code for switching backend between worker-shell and container, and warns that exitCode 0 alone is not durability — sync.status must be 'complete'. A cost model projects that dropping container duty cycle from 100% to 10% reduces monthly cost from ~$36.83 to ~$7.53 (79.6%), while node_modules is deliberately kept disposable. A strong read for engineers building coding agents, sandboxes, or durable workspaces.

x.com · 10 min · Agent Infrastructure · Cloudflare · Cost Optimization

08-03

Improving token efficiency in GitHub Agentic Workflows

GitHub's team instrumented its own fleet of Agentic Workflows through an API proxy, emitting a normalized token-usage.jsonl per run and building two daily agentic workflows—a usage Auditor and an Optimizer—that read those logs, flag anomalies, and file concrete optimization issues. The biggest wins came from pruning unused MCP tool registrations (each request can carry 10–15KB of schema overhead), replacing GitHub MCP calls with deterministic GitHub CLI invocations, and moving fixed data-gathering into pre-agentic setup steps. To compare across models they define Effective Tokens: ET = m×(1.0×I + 0.1×C + 4.0×O) with model multipliers. Of 12 production workflows, nine received optimizer changes; measured reductions were 62% for Auto-Triage Issues (109 runs), 43% for Security Guard, and 59% for Smoke Claude. One workflow regressed 5% due to workload shift, and one misconfigured bash allowlist caused a 64-turn fallback loop. The post argues for episode- and portfolio-level efficiency analysis.

github.blog · 18 min · Agent Engineering · CLI · Cost Optimization

07-25

Introducing Claude Opus 5: Near-Frontier Performance at Half the Cost

Anthropic releases Claude Opus 5, approaching the frontier intelligence of Fable 5 at half the cost. It achieves state-of-the-art results on coding and knowledge work benchmarks, e.g., Frontier-Bench v0.1 (more than double Opus 4.8 performance) and ARC-AGI 3 (3x score over next best). However, it remains behind Mythos 5 on cybersecurity tasks. The model offers tunable effort settings for cost-performance trade-offs. Customer reports highlight gains in software engineering, finance, and legal work. Alignment improves, but cybersecurity capabilities are intentionally limited. Safety classifiers intervene ~85% less than Fable 5. Pricing same as Opus 4.8, with fast mode available.

www.anthropic.com · 20 min · AI Engineering · Anthropic · Cost Optimization

07-20

Kimi K3 vs Claude Fable 5 vs GPT-5.6: Task-Based Decision Guide

As of July 2026, no single model dominates all tasks. Kimi K3 (2.8T params) leads frontend UI and image understanding, winning 6 of 7 domains at 1/12 the cost of Fable 5. Claude Fable 5 achieves 80.3% on SWE-Bench Pro, ideal for backend architecture and long-running autonomous agents, but at $10/$50 per million tokens. GPT-5.6 Sol excels at debugging but may game vague success criteria. This guide provides a task-specific routing framework, cost math, licensing considerations, and vendor lock-in mitigation. The key skill is routing, not picking a permanent favorite.

x.com · 25 min · Agent Engineering · AI · Cost Optimization

07-17

Fable's judgement

Simon Willison shares a practical tip from the Claude Code team: let Fable use its own judgement to decide when to write tests and delegate coding tasks to cheaper subagents. With Claude Code's Fable token prices about to rise, he demonstrates how to configure a memory file so the main model can autonomously pick a lower-cost model (Sonnet for substantial work, Haiku for trivial edits) for implementation tasks while retaining judgement-heavy work in Fable. Early results show significantly reduced Fable consumption without sacrificing productivity.

simonwillison.net · 2 min · Agent Engineering · Ai Tooling · Claude Code

07-12

Anthropic's Goodwill Drain: Lock-in, Price Hikes, and an Engineer's Reckoning

A firsthand account of Anthropic's deteriorating trust: unreliable APIs tied to subscriptions, a locked-in and buggy Claude Code ecosystem, and opaque billing changes that effectively raise costs. The author argues these moves fund model training, not product improvement. Their response: shift to an 'agent-assisted' workflow and swap Claude for open-source models like Qwen and GLM via OpenRouter, gaining cost control and data security. For engineers feeling trapped in a single AI platform.

raheeljunaid.com · 11 min · Agent Engineering · Anthropic · Claude Code

07-04

Superpowers 6: Cutting Build Cost 60% via Autoresearch Loop

Superpowers 6 is released, with its biggest improvements driven by an automated research loop. The author used Anthropic's Fable model (briefly available) to systematically optimize their Subagent Driven Development pipeline. Over 36 hours and ~$165 in token spend, 25 experiments were run, yielding a 50% reduction in wall-clock time and 60% reduction in token consumption vs. v5. Key optimizations: merging spec compliance and code review agents, pre-baking review packets to minimize git operations, and dynamic agent allocation based on task type (e.g., using cheap haiku for non-code plans). The post also documents falsified hypotheses (e.g., capping controller thinking backfires) and emphasizes the role of their eval suite in rigorous measurement.

blog.fsck.com · 8 min · Agent Engineering · AI Engineering · Anthropic

07-03

The Claude Opus 4.8 Setup Guide: How to Get Maximum Quality for Minimum Cost (Exact Config Inside)

A hands-on configuration guide published day after Claude Opus 4.8's release. The core value lies not in benchmark improvements (SWE-bench 87.6% → 88.6%) but in three operational features: Effort Control for per-task reasoning depth, Fast Mode at 3x cheaper than before, and Dynamic Workflows supporting up to 1,000 parallel subagents. The author provides a cost-optimization matrix routing tasks to Haiku/Sonnet/Opus at different effort levels, claiming ~50% monthly savings ($400-600 down to ~$205) for heavy users. Includes copy-paste configs for environment variables and settings.json. Practical for Claude Code users focused on cost control, though the savings claims are unverified estimates.

x.com · 9 min · Agents · Ai Tooling · Claude Code

07-01

Micro-Agent: Beat Frontier Models with Collaboration inside Model API

The vLLM Semantic Router proposes a different take: a router is not just a request dispatcher but an amplifier of model capability. The core idea is to encapsulate multi-model collaboration inside a single model API call. The user sees one model endpoint (vllm-sr/auto), but behind it the router can automatically select a collaboration pattern — from cost-aware escalation (Confidence), parallel aggregation (Ratings), repeated mixture-of-model reasoning (ReMoM), disagreement-as-signal (Fusion), to budgeted micro-agent workflows (Workflows). These patterns are controlled, configurable, observable runtimes, not application glue. Benchmarks on LiveCodeBench, GPQA-Diamond, and Humanity's Last Exam show the closed-model collaboration scheme (VSR Closed) achieving 92.6%, 96.0%, and 50.0% respectively, matching or beating single frontier models like Fugu Ultra and GPT-5.5. This article is valuable because it sinks multi-model collaboration from the product or application layer down to the serving infrastructure layer, while preserving a single model identity. For engineers building inference routing, multi-model strategies, or cost optimization.

vllm.ai · 14 min · AI Engineering · Cost Optimization · LLM

06-27

The New Software Lifecycle: From Writing Code to Judging It

Key insights from a Google whitepaper on how AI transforms the software lifecycle. The core thesis: an agent is 10% model and 90% harness (instructions, tools, sandboxes, orchestration, observability). Context engineering is the primary cost lever, with a critical distinction between static context (loaded every turn, reliable but expensive) and dynamic context (loaded on demand, cost-efficient but needs careful design). Verification determines whether you're vibe coding or doing agentic engineering: tests for deterministic parts, evals for non-deterministic output and trajectory. Real data: one team moved a coding agent from outside top 30 to top 5 on Terminal Bench 2.0 by changing only the harness with the same model; LangChain added 13.7 points on the same benchmark by changing system prompt, tools, and middleware around a fixed model. Implementation collapses from weeks to hours, while specification and verification become the new bottlenecks. For engineers and tech leads adopting AI agents in production workflows.

addyosmani.com · 15 min · Agent Architecture · AI Engineering · Context Engineering

06-24

Loop Engineering: How One Loop Ships 259 PRs a Month

This article breaks down the engineering of AI-driven development loops, contrasting a single engineer shipping 259 PRs in a month with a runaway loop that burned $47,000. It dissects six essential components—state file, automation/scheduling commands (e.g., /loop, /schedule, /goal), git worktrees, skills, MCP connectors, and sub-agents (writer vs. checker)—with concrete configuration examples for both Claude Code and OpenAI Codex. The piece provides a brake configuration template (max_turns, max_budget_usd, scope, circuit_breaker), describes four failure modes, and offers low-cost starting strategies. Aimed at engineers building or evaluating AI agent workflows.

x.com · 12 min · Agent Engineering · Ai Tooling · Claude Code

06-24

Agent Loops for PMs: The Hard Part Is the Stop Condition

This article explains agent loops for product managers, distinguishing between routines, workflows, and true goal-driven loops. The key insight is that the hard part is defining the stop condition: a verifiable definition of done with an objective check or independent grader. It provides a template for building loops, guidance on writing stop conditions, and cost management advice (e.g., tracking cost per accepted change). Common failure modes are discussed: runaway costs without iteration caps, context drift, and passing tests without being correct. The author concludes that loop engineering is just the latest name for intent engineering—precisely defining goals, boundaries, and completion criteria.

x.com · 12 min · Agent Engineering · Cost Optimization · Product Management

06-24

Agent Loops for PMs: The Hard Part Is the Stop Condition

x.com · 12 min · Agent Engineering · Cost Optimization · Product Management

06-23

Old Software Was Fast Because It Had No Choice

The article argues that modern software has become bloated not because of any single bad decision, but because hardware is too easy to provision. Using the example of a Java component launching a Spark cluster, the author points out that engineers routinely add memory and CPU 'just in case,' and these temporary patches harden into defaults. The JVM reads an inflated container limit and grows its heap, GC gets lazier, and resources are silently wasted. The real problem is that cost moves from the decision-maker to someone else—the person adding a dependency today is not the one debugging it tomorrow. The solution is explicit resource budgets that force teams to justify any increase in footprint. Recommended for backend engineers, SREs, and platform teams running services in the cloud.

yusufaytas.com · 9 min · Cloud Native · Cost Optimization · Java

06-20

Stop building Foxconn factories for your agents

Garry Tan reflects on his experience building a 540,000-line Rails app, using the Foxconn factory as a metaphor for the dominant AI agent development pattern: wrapping hyper-intelligent models in mountains of code, tests, and guardrails. He argues the economics have inverted—model calls are now cheap and the models are smarter, making the old instinct to ration and control them obsolete. The new paradigm is 'just-in-time software' and 'skill packs,' where lean markdown instructions and minimal TypeScript replace bloated engineering frameworks. A concrete example shows a hackathon judge agent built in an afternoon, doing what previously required a full software project. The essay challenges engineers to abandon the 2013 mental model of measuring capability by lines of code and to embrace 'tokenmaxxing' to gain a 2-3 year competitive advantage. It is aimed at engineers who are coding with AI but still trapped by traditional software metrics and mistrustful architectures.

x.com · 14 min · Agents · Ai Tooling · Code

06-16

The Context Compression Layer for AI Agents: 60–95% Fewer Tokens, Zero Accuracy Loss

Headroom is a local-first context compression layer for AI agents that slashes token usage from tool outputs, logs, files, and RAG chunks by 60–95% before they reach the LLM, with preserved accuracy. It offers library, proxy, MCP server, and agent wrapper modes, using a content router to select the best compressor for JSON, code, or prose. Reversible compression ensures originals are retrievable on demand. With cross-agent memory and `headroom learn` for mining failed sessions, it is ideal for engineers running coding agents daily and anyone seeking to slash LLM costs without changing their workflow.

github.com · 18 min · Agent Architecture · Ai-Memory · Context Engineering

06-09

How to Design a Loop That Prompts Your Agent

This article presents a loop architecture that enables an AI agent to autonomously complete multi-step tasks by building an automated prompting system instead of manually crafting each prompt. It breaks down the loop into five parts: defining a 'done' check, building prompts from dynamic state rather than hand-fed instructions, executing actions while capturing all outputs, feeding failures back as the next prompt, and setting hard stop conditions like max turns and cost limits. A walkthrough of fixing a login bug shows the loop in action, emphasizing that real costs come from repeated turns, making guardrails critical. Encapsulating repeated operations into reusable skills is highlighted as the key to long-term value, and common mistakes—like lacking an exit condition or discarding error output—are pointed out. Suitable for developers shifting from one-shot prompts to designing agent control flows.

x.com · 18 min · Agent Architecture · Agents · AI Engineering

06-07

Weekly AI Roundup: Claude Limits Doubled, SpaceX IPO, Microsoft Model Data Contradiction

A roundup of 10 major AI and tech news items from the first week of June 2026. MiniMax M3 was released, beating GPT-5.5 on coding benchmarks at $0.6/M tokens, though independent verification is pending. DeepSeek raised ~$7.4B in its first external funding round, while Unitree completed its IPO review in a record 73 days. Kimi Work, Coze 3.0, and Qwen3.7-Plus all launched new Agent capabilities. Doubao announced subscription plans. ChatGPT surpassed 1 billion monthly active users. Anthropic doubled Claude Cowork's usage limits, secretly filed for an IPO, and published a report stating Claude writes 80% of its own code. NVIDIA unveiled the ARM-based RTX Spark at Computex. SpaceX is set to IPO on June 12, with Google disclosed paying $920M/month for compute. Microsoft's MAI-Thinking-1 faced backlash after its claimed 'clean data' was revealed to include Common Crawl, and GitHub Copilot's switch to metered billing caused developer bills to spike.

mp.weixin.qq.com · 7 min · AI Engineering · AI Industry · Cost Optimization