Glean 拾遗
中 EN
Submit a link
Daily · timeline

A few picks a day.

Mon, Aug 10, 2026 3picks
← 08-09
Calendar ▾
2026 · 08
MoTuWeThFrSaSu ·····12345678910111213141516171819202122232425262728293031
has picks today
06:00

From Vibes to Data-Driven Development: The AI Engineer Mindset

AI 工程师心态：接受概率，用可量化标准替代“感觉”

This article argues that building with LLMs requires a fundamental mindset shift from deterministic to probabilistic systems. The author introduces two concepts: the 'Vibes-Only Trough' (where demos appear to work but lack real-world coverage) and the 'Data-Driven Slope' (where systematic evaluation and user data guide improvements). Success criteria must be specific and measurable, such as 92% agreement with human labels for sentiment analysis, 95% of classifications within 500ms, and 90% match on high-confidence predictions. The key practice is collecting data from every user interaction - starting with simple thumbs up/down feedback and evolving into richer pipelines. The article honestly states that first versions usually fail due to insufficient data, and that performance volatility is inherent to probabilistic systems. The trick is tracking changes systematically and building a culture of iterative improvement based on real metrics. Aimed at engineers moving LLM apps to production and teams needing an evaluation-first culture.

www.aihero.dev · 6 min · AI Engineering · LLM · LLM Evaluation
06:00

Your App Is Only As Good As Its Evals

AI 应用的质量上限，取决于你的 Evals

In LLM apps, inputs no longer map to deterministic outputs—any small change can remake the entire behavior. This post argues that evals are the AI engineer's unit tests, the primary way to impose predictability on a probabilistic system. It walks through three evaluation types: deterministic assertions, human review, and LLM-as-a-judge, noting the cost trade-offs and recommending split cadences (fast local set vs. daily full set). It then explains the data flywheel: converting user downvotes into new eval cases and feeding them back into the system. Tooling compares Braintrust, a cloud eval platform, with Evalite, the author's Vitest-based local runner. Useful for engineers moving LLM prototypes to production.

www.aihero.dev · 9 min · AI Engineering · Evals · LLM · LLM-as-a-Judge
06:00

17 Techniques for Improving Your LLM-Powered App

从改提示词到微调：一条按成本排序的 LLM 应用优化阶梯

This guide walks through 17 techniques for improving LLM-powered applications, deliberately ordered from cheapest to most expensive on what the author calls the Staircase of Complexity Hell. It starts with prompt tweaks — role prompting, XML tags, structured outputs, chain-of-thought, multishot, temperature, and tool calling — before moving to system-level patterns: RAG, chunking, agentic loops, parallelization, evaluator-optimizer, LLM routers, and fine-tuning. The core advice: work down the staircase only after simpler options are exhausted. Concrete evidence includes Claude 4.6 returning 400 on prefilling, parallel processing cutting 10-document analysis from ~10s to 2-3s, routers bypassing the common ~30-tool limit per model, and fine-tuned smaller models beating larger ones on specific tasks. Each section names real trade-offs, such as CoT increasing latency and agentic loops adding decision overhead. Useful as a practical checklist for engineers iterating on LLM systems.

www.aihero.dev · 23 min · Agent Engineering · AI Engineering · LLM · Prompt Engineering · Rag