标签 · Performance — Glean

Recent picks

16picks · chronological

09-01

Zod 4.5: z.compile() precompilation speeds parse up to 9x, slashes memory

Zod 4.5 introduces z.compile(), an AOT schema compiler that speeds up parsing of objects, arrays, and unions by ~3–9x. Combined with z.validate()—a fast boolean check that skips ZodError construction—invalid inputs are up to 16x cheaper to reject than safeParse. Benchmarks show 47.5M ops/s on the Moltar parseSafe fixture, ahead of typia's 45.3M. A move of bound methods to prototypes cuts per-schema heap retention by up to 9.8x. The release also includes breaking changes: string length now counts Unicode code points, z.iso.datetime() requires seconds, record keys and intersections match TypeScript semantics, and __proto__ is always stripped. Useful for TS teams doing heavy runtime validation in services, forms, and data pipelines.

zod.dev · 26 min · Developer Tools · Open Source · Performance

08-10

5 Questions to Ask Before Choosing an LLM

Choosing an LLM is not a one-time decision—it must be revisited as models and your app evolve. This guide breaks the choice into five questions: open vs. closed source, cost, latency, performance, and context window. Open models require self-hosting or an API provider like Hugging Face or Groq; closed models are hosted by vendors and priced per token. Latency should be measured with TTFT and TPOT. Benchmarks like Chatbot Arena and Open LLM Leaderboard are useful early indicators, but prone to overfitting—the only real test is running your own evals within your application. Reasoning models such as o1 excel at planning-heavy tasks but cost more and respond slower. Context windows count both input and completion tokens, which is why RAG chunking exists. A practical primer for developers starting model selection.

www.aihero.dev · 8 min · AI Engineering · Context Engineering · Cost Optimization

08-06

Jeff Dean's Last Interview: AI Speed, the 0% Rule, and His Google Exit

In his last interview before leaving Google, Jeff Dean admits his year-old prediction undershot reality: models are improving faster than expected at complex tasks, and agentic systems can now run continuously for weeks. He likens specialized inference hardware to the 2001 'put the search index in RAM' moment, citing 50x latency and 30-80x energy-efficiency gains. A single compute op costs ~1 picojoule; moving data costs 1000x more, which forces batching. The conversation covers context engineering as the new frontier, the napkin math behind TPU, and a founder heuristic: test your domain with the strongest general model — 0% or 1% success is a good sign, 20% is a red flag. Once agents write all the code, he argues, taste — knowing what to have agents solve — becomes the scarce skill. Relevant for AI infrastructure engineers, agent builders, and AI founders.

www.infoq.cn · 14 min · AI Agents · Context Engineering · Inference Hardware

07-28

How 5 Tools + Custom Hooks Cut Claude Code Token Usage by 90%+

A practical guide to cutting Claude Code token usage by over 90% using five layered tools and custom Hooks. CBM replaces file-greping with knowledge graph queries (99.2% token reduction), context-mode extends 200K-window sessions from 30min to 3h+, RTK compresses shell output, Headroom compresses the full API payload before sending, and Caveman trims verbose replies. Enforcement Hooks ensure each layer is used by default, reducing code search tokens from ~400K to ~3.4K.

x.com · 10 min · Agents · AI · Performance

07-25

Introducing Claude Opus 5: Near-Frontier Performance at Half the Cost

Anthropic releases Claude Opus 5, approaching the frontier intelligence of Fable 5 at half the cost. It achieves state-of-the-art results on coding and knowledge work benchmarks, e.g., Frontier-Bench v0.1 (more than double Opus 4.8 performance) and ARC-AGI 3 (3x score over next best). However, it remains behind Mythos 5 on cybersecurity tasks. The model offers tunable effort settings for cost-performance trade-offs. Customer reports highlight gains in software engineering, finance, and legal work. Alignment improves, but cybersecurity capabilities are intentionally limited. Safety classifiers intervene ~85% less than Fable 5. Pricing same as Opus 4.8, with fast mode available.

www.anthropic.com · 20 min · AI Engineering · Anthropic · Cost Optimization

07-24

Stop Installing Plugins on Claude Code: Everything You Need Is Built In

The author installed 23 Claude Code plugins, found baseline overhead of 62,000 tokens before the first prompt. After removing almost all, session length tripled, output quality improved, and token spend halved. This post walks through built-in commands that replaced each plugin: /context for token diagnostics, /compact for conversation compression, /resume for session recovery, Esc Esc for checkpoint restore, /cost for real-time spending, /model for switching models mid-session, /init for project setup, /review for code audits, and custom /.claude/commands/ for zero-overhead workflows. Only three extensions survived the purge, reducing baseline overhead to ~6k tokens. Actionable for any Claude Code user slowed by plugins.

x.com · 16 min · Agents · LLM · Performance

07-13

Your Worker can now have its own cache in front of it

Cloudflare launched Workers Cache, a tiered cache that sits in front of your Worker, enabled by a single line in wrangler.jsonc (`"cache": { "enabled": true }`). On a cache hit, the Worker does not run and CPU time is not billed; on a miss, the Worker runs and populates the cache. It supports stale-while-revalidate, Vary-based content negotiation, and multi-tenant isolation via ctx.props. Unlike zone-level caches, Workers Cache belongs to the Worker itself, working on custom domains, workers.dev, and Workers for Platforms. Most notably, the cache sits in front of every Worker entrypoint, including those invoked via service bindings and ctx.exports, enabling developers to compose an app as a chain of entrypoints with per-entrypoint caching control. Framework integrations (including Astro) are available.

blog.cloudflare.com · 32 min · Caching · Cloudflare Workers · Edge Computing

06-23

Old Software Was Fast Because It Had No Choice

The article argues that modern software has become bloated not because of any single bad decision, but because hardware is too easy to provision. Using the example of a Java component launching a Spark cluster, the author points out that engineers routinely add memory and CPU 'just in case,' and these temporary patches harden into defaults. The JVM reads an inflated container limit and grows its heap, GC gets lazier, and resources are silently wasted. The real problem is that cost moves from the decision-maker to someone else—the person adding a dependency today is not the one debugging it tomorrow. The solution is explicit resource budgets that force teams to justify any increase in footprint. Recommended for backend engineers, SREs, and platform teams running services in the cloud.

yusufaytas.com · 9 min · Cloud Native · Cost Optimization · Java

06-07

The Orchestration Tax: Why 20 AI Agents Don't Mean 20x Output

Addy Osmani coins 'orchestration tax': spawning agents is cheap, but closing the loop—reviewing, judging, merging—is serial and bounded by your cognitive bandwidth. Using Amdahl's Law and Python's GIL as analogies, he argues you are the single-threaded bottleneck in an otherwise concurrent system. Tactics: cap parallelism to your review rate, split work into delegable vs. judgment-heavy piles, batch reviews to cut context-switch costs, and force agents to self-verify. Aimed at engineers who run multiple AI agents daily and feel busy but unproductive.

x.com · 9 min · Agents · Performance

06-06

Lessons from Building Claude Code: Prompt Caching Is Everything

Anthropic engineer shares hard-won lessons from optimizing prompt caching in Claude Code. Prompt caching relies on strict prefix matching, so the order of static vs dynamic content is critical — static system prompts, tools, and context must come first. The post reveals counterintuitive pitfalls: don't update the system prompt mid-conversation (pass updates via messages instead), never switch models or modify tool sets mid-session (it invalidates the entire cache), and when compacting context, reuse the parent conversation's prefix to avoid paying full price for tokens. Practical patterns include using tools like EnterPlanMode to model state transitions, deferring tool loading, and running alerts on cache hit rate. A must-read for anyone building long-running agentic products.

x.com · 8 min · Agents · LLM · Performance

06-05

What Really Differentiates LLMs Happens After Pretraining: A Full Post-Training Pipeline Breakdown

A comprehensive deep-dive into the full LLM training pipeline, arguing that the real capability gap in 2026 lies not in pretraining but in the post-training stack: instruction tuning, RL, reward design, Agent training, and distillation. The article breaks down the end-to-end process step-by-step — from data recipes and system architecture constraints, through the four-stage post-training pipeline (Cold Start SFT → GRPO-based Reasoning RL → Rejection Sampling FT → Alignment RL), Grader/Reward evaluation loops, Agent training with PARL and Meta-Harness, to distillation and deployment. Key engineering insights include DeepSeek-R1's public recipe, why GRPO simplifies PPO by removing the value network, PRM vs ORM trade-offs, and the shift from optimizing answers to optimizing harness programs. Targeted at engineers who want to trace concrete capability gains back to specific training stages.

tw93.fun · 27 min · Agents · LLM · Performance

06-04

How to Stop Hitting Claude Usage Limits: 23 Token-Saving Habits

A personal guide of 23 habits to reduce Claude token usage, based on author's experience and Anthropic docs. Includes converting files before upload, planning in Chat before building files, using edit instead of follow-ups, and voice-to-text for richer prompts. Helps paid users go from daily limits to hitting them once a month. For heavy Claude users.

x.com · 17 min · AI · Performance

05-31

Building self-improving tax agents with Codex

OpenAI and Thrive Holdings co-developed Tax AI for Crete's accounting firms, using a Codex-driven self-improvement loop. The system processed 7,000 returns with 97% accuracy and 50% throughput increase, cutting one senior accountant's prep time from 180 to 15 hours. The design relies on three pillars: practitioner feedback, production traces, and a Codex iteration cycle. A detailed rental property example shows how practitioner corrections become eval targets, then Codex investigates and proposes fixes. Practical for teams building self-improving agents in expert domains.

openai.com · 15 min · Agents · AI · Performance

05-29

ClickHouse 10 Best Practices

A ClickHouse solution architect shares 10 field-tested best practices derived from customer engagements, covering schema design, data types, partitioning, skipping indexes, JSON type, data ingestion, materialized views, system tables, ReplacingMergeTree, and JOIN optimization. Benchmarks on a 150M-row Amazon reviews dataset quantify the impact: proper ORDER BY reduces rows scanned by 347×, unnecessary partitioning slows queries by 46×, correct data types cut storage by 12% and double query speed, skipping indexes reduce scans by 80%, and dictionary lookups beat regular JOINs by nearly 3×. The article emphasizes understanding ClickHouse internals to achieve orders-of-magnitude improvements without hardware changes.

www.infoq.cn · 15 min · Database · Performance

05-29

Introducing React Best Practices: A Structured Repo for AI Agents

Vercel distills 10+ years of React and Next.js optimization into a structured repo with 40+ rules across 8 categories, ordered by impact from eliminating waterfalls to JavaScript micro-optimizations. Each rule includes code samples and impact ratings, and compiles into an AGENTS.md document consumable by AI coding agents.

vercel.com · 6 min · AI · Performance · React

05-29

From OTel to Rotel: 4x Throughput Increase in PB-Scale Tracing

This article benchmarks OpenTelemetry data planes for writing trace spans to ClickHouse. On the same 8‑core host, Rotel achieves 3.7 million spans/sec (462k spans/core/sec), a >4× improvement over the OTel Collector. Gains come from three optimizations: binary encoding of JSON columns in RowBinary, moving deserialization to a shared thread pool to avoid tokio blocking and glibc allocator lock contention, and enabling fast LZ4 compression. The test also exposes silent data loss in the OTel Collector under backpressure. For engineers scaling large telemetry pipelines.