标签 · Verification — Glean

Recent picks

4picks · chronological

11-09

The Harness Is the Machine: Task Contracts, Worktrees, Receipts

A long-form X post outlining a production architecture for coding agents. The thesis: when an agent fails a multi-file refactor, the problem is not the prompt or the model but the environment around it — the harness. The author lays out a seven-stage pipeline: Opus compiles a task_contract.json with path allowlists and a cost/turn/timeout budget; the harness spawns an isolated git worktree; Sonnet edits only inside the boundary; Haiku validates every tool call parameter; a separate model acts as adversarial skeptic against the diff; a pre-commit hook checks test exit codes and signs a receipt.json; failures crystallize into rules, skills, and regression evals. Includes contract schema, isolate_worktree.sh, pre-tool.sh, receipt format, and a 10-point pre-flight checklist. Caveat: the Opus/Sonnet/Haiku/Fable 5.5 model names and the 18%→95% success numbers are unsourced, and the post ends with a Telegram funnel link.

x.com · 20 min · Agent Engineering · Agent Harness · Claude Code

11-09

Plan With Code, Not Prose: pstack Pt. 2

Part two of poteto's pstack guide covers the front half of the agent workflow: research, planning, prototyping, and architecture. She argues agents mostly fail for two reasons — under-specified intent and missing context — so she has the agent restate a reported problem in its own words before touching code, then uses /how, /why, /recall, and /teach to rebuild runtime mechanics and historical motivation from git history, PR comments, Linear, Slack, Datadog, and Sentry. Planning happens in code rather than prose: README-driven development, a tutorial written before the API, throwaway prototypes driven by /control-app with screenshots and timing measurements, and /architect, which spins up parallel design candidates across model families and has a different model cross-judge them. Escape hatches like any, forced casts, or the same workaround at unrelated call sites count as proof the design is wrong. Aimed at engineers already shipping agent-written code to production.

x.com · 20 min · Agent Orchestration · Agent Workflows · Agentic Coding

11-09

The Complete Guide to pstack, Pt. 1: Verification Is All You Need

poteto argues that a verification skill — an agent's ability to drive the app, screenshot it, read traces and confirm its own change — is critical infrastructure rather than a nice-to-have, because it takes the human out of the loop. The post walks through /create-verification-skill: wrap dev-environment control in an agent-friendly CLI (doctor, snapshot, screenshot, wait-settle, feature-flag), then catalog every feature in a markdown Feature Map describing what it does, how a user reaches it, and known gotchas. She advises against scaling with git worktrees (roughly 10 agents per machine before resources bite) and recommends Cursor cloud agents plus /swarm for parallel verification runs. Wired into a scheduled routine, the skill can auto-reproduce user bug reports from Slack. The CLI design rules — --dry-run on destructive commands, descriptive errors, JSON output, progressive disclosure via subcommands — are the most transferable part. Claims of 2,000 PRs/month and 100-1000x output come with no measurement.

x.com · 19 min · Agent Harness · Agent Skills · Agent Tool Design

06-27

The New Software Lifecycle: From Writing Code to Judging It

Key insights from a Google whitepaper on how AI transforms the software lifecycle. The core thesis: an agent is 10% model and 90% harness (instructions, tools, sandboxes, orchestration, observability). Context engineering is the primary cost lever, with a critical distinction between static context (loaded every turn, reliable but expensive) and dynamic context (loaded on demand, cost-efficient but needs careful design). Verification determines whether you're vibe coding or doing agentic engineering: tests for deterministic parts, evals for non-deterministic output and trajectory. Real data: one team moved a coding agent from outside top 30 to top 5 on Terminal Bench 2.0 by changing only the harness with the same model; LangChain added 13.7 points on the same benchmark by changing system prompt, tools, and middleware around a fixed model. Implementation collapses from weeks to hours, while specification and verification become the new bottlenecks. For engineers and tech leads adopting AI agents in production workflows.

addyosmani.com · 15 min · Agent Architecture · AI Engineering · Context Engineering