标签 · Performance — Glean

Recent picks

10picks · chronological

07-13

Your Worker can now have its own cache in front of it

Cloudflare launched Workers Cache, a tiered cache that sits in front of your Worker, enabled by a single line in wrangler.jsonc (`"cache": { "enabled": true }`). On a cache hit, the Worker does not run and CPU time is not billed; on a miss, the Worker runs and populates the cache. It supports stale-while-revalidate, Vary-based content negotiation, and multi-tenant isolation via ctx.props. Unlike zone-level caches, Workers Cache belongs to the Worker itself, working on custom domains, workers.dev, and Workers for Platforms. Most notably, the cache sits in front of every Worker entrypoint, including those invoked via service bindings and ctx.exports, enabling developers to compose an app as a chain of entrypoints with per-entrypoint caching control. Framework integrations (including Astro) are available.

blog.cloudflare.com · 32 min · Caching · Cloudflare Workers · Edge Computing

06-23

Old Software Was Fast Because It Had No Choice

The article argues that modern software has become bloated not because of any single bad decision, but because hardware is too easy to provision. Using the example of a Java component launching a Spark cluster, the author points out that engineers routinely add memory and CPU 'just in case,' and these temporary patches harden into defaults. The JVM reads an inflated container limit and grows its heap, GC gets lazier, and resources are silently wasted. The real problem is that cost moves from the decision-maker to someone else—the person adding a dependency today is not the one debugging it tomorrow. The solution is explicit resource budgets that force teams to justify any increase in footprint. Recommended for backend engineers, SREs, and platform teams running services in the cloud.

yusufaytas.com · 9 min · Cloud Native · Cost Optimization · Java

06-07

The Orchestration Tax: Why 20 AI Agents Don't Mean 20x Output

Addy Osmani coins 'orchestration tax': spawning agents is cheap, but closing the loop—reviewing, judging, merging—is serial and bounded by your cognitive bandwidth. Using Amdahl's Law and Python's GIL as analogies, he argues you are the single-threaded bottleneck in an otherwise concurrent system. Tactics: cap parallelism to your review rate, split work into delegable vs. judgment-heavy piles, batch reviews to cut context-switch costs, and force agents to self-verify. Aimed at engineers who run multiple AI agents daily and feel busy but unproductive.

x.com · 9 min · Agents · Performance

06-06

Lessons from Building Claude Code: Prompt Caching Is Everything

Anthropic engineer shares hard-won lessons from optimizing prompt caching in Claude Code. Prompt caching relies on strict prefix matching, so the order of static vs dynamic content is critical — static system prompts, tools, and context must come first. The post reveals counterintuitive pitfalls: don't update the system prompt mid-conversation (pass updates via messages instead), never switch models or modify tool sets mid-session (it invalidates the entire cache), and when compacting context, reuse the parent conversation's prefix to avoid paying full price for tokens. Practical patterns include using tools like EnterPlanMode to model state transitions, deferring tool loading, and running alerts on cache hit rate. A must-read for anyone building long-running agentic products.

x.com · 8 min · Agents · LLM · Performance

06-05

What Really Differentiates LLMs Happens After Pretraining: A Full Post-Training Pipeline Breakdown

A comprehensive deep-dive into the full LLM training pipeline, arguing that the real capability gap in 2026 lies not in pretraining but in the post-training stack: instruction tuning, RL, reward design, Agent training, and distillation. The article breaks down the end-to-end process step-by-step — from data recipes and system architecture constraints, through the four-stage post-training pipeline (Cold Start SFT → GRPO-based Reasoning RL → Rejection Sampling FT → Alignment RL), Grader/Reward evaluation loops, Agent training with PARL and Meta-Harness, to distillation and deployment. Key engineering insights include DeepSeek-R1's public recipe, why GRPO simplifies PPO by removing the value network, PRM vs ORM trade-offs, and the shift from optimizing answers to optimizing harness programs. Targeted at engineers who want to trace concrete capability gains back to specific training stages.

tw93.fun · 27 min · Agents · LLM · Performance

06-04

How to Stop Hitting Claude Usage Limits: 23 Token-Saving Habits

A personal guide of 23 habits to reduce Claude token usage, based on author's experience and Anthropic docs. Includes converting files before upload, planning in Chat before building files, using edit instead of follow-ups, and voice-to-text for richer prompts. Helps paid users go from daily limits to hitting them once a month. For heavy Claude users.

x.com · 17 min · AI · Performance

05-31

Building self-improving tax agents with Codex

OpenAI and Thrive Holdings co-developed Tax AI for Crete's accounting firms, using a Codex-driven self-improvement loop. The system processed 7,000 returns with 97% accuracy and 50% throughput increase, cutting one senior accountant's prep time from 180 to 15 hours. The design relies on three pillars: practitioner feedback, production traces, and a Codex iteration cycle. A detailed rental property example shows how practitioner corrections become eval targets, then Codex investigates and proposes fixes. Practical for teams building self-improving agents in expert domains.

openai.com · 15 min · Agents · AI · Performance

05-29

ClickHouse 10 Best Practices

A ClickHouse solution architect shares 10 field-tested best practices derived from customer engagements, covering schema design, data types, partitioning, skipping indexes, JSON type, data ingestion, materialized views, system tables, ReplacingMergeTree, and JOIN optimization. Benchmarks on a 150M-row Amazon reviews dataset quantify the impact: proper ORDER BY reduces rows scanned by 347×, unnecessary partitioning slows queries by 46×, correct data types cut storage by 12% and double query speed, skipping indexes reduce scans by 80%, and dictionary lookups beat regular JOINs by nearly 3×. The article emphasizes understanding ClickHouse internals to achieve orders-of-magnitude improvements without hardware changes.

www.infoq.cn · 15 min · Database · Performance

05-29

Introducing React Best Practices: A Structured Repo for AI Agents

Vercel distills 10+ years of React and Next.js optimization into a structured repo with 40+ rules across 8 categories, ordered by impact from eliminating waterfalls to JavaScript micro-optimizations. Each rule includes code samples and impact ratings, and compiles into an AGENTS.md document consumable by AI coding agents.

vercel.com · 6 min · AI · Performance · React

05-29

From OTel to Rotel: 4x Throughput Increase in PB-Scale Tracing

This article benchmarks OpenTelemetry data planes for writing trace spans to ClickHouse. On the same 8‑core host, Rotel achieves 3.7 million spans/sec (462k spans/core/sec), a >4× improvement over the OTel Collector. Gains come from three optimizations: binary encoding of JSON columns in RowBinary, moving deserialization to a shared thread pool to avoid tokio blocking and glibc allocator lock contention, and enabling fast LZ4 compression. The test also exposes silent data loss in the OTel Collector under backpressure. For engineers scaling large telemetry pipelines.