Why Harness Engineering Is So Hard

By @WinterArc2125 · 2026-07-25T15:42:52.000Z

Five months, 104 commits, and one repeating lesson: harness engineering is hard.

There is an illusion that settles over almost every project built on a large language model. The illusion is that the model is the product. You wire up an API call, write a prompt, get something impressive in the first demo, and feel as though the hard part is finished.

It is not.

The difficult part is turning that impressive demonstration into behaviour you can rely on. It is getting the model to follow the same constraints across different inputs, express the right ideas without drifting, survive model updates, fail visibly instead of silently, and produce output that can be tested even though it will never be perfectly deterministic.

That behavioural layer is a large part of what I mean by the harness: the prompts, examples, schemas, validators, evals, and guardrails that sit between a probabilistic model and the person using your product. The model generates the output. The harness is what makes that output usable.

This is why harness engineering is so hard, and why the difficulty keeps surprising people who are otherwise excellent engineers. I've spent months in it, and most of the pain comes from a handful of structural facts about working with a probabilistic system that look, on the surface, like ordinary software problems, but to our dismay, are not.

You can't write the test you want to write

The first thing that breaks is your testing instinct. Every testing practice you've built up over a career assumes determinism: given this input, the code produces this output, and I assert equality. But feed the same input to a language model twice, and you get two different outputs. Not always wildly different, sometimes the difference is a few words, a reordered clause, a slightly different way of phrasing the same idea, but different enough that a strict equality assertion is useless.

This sounds like a minor annoyance. It is not. It removed the floor under my whole quality strategy for my specific harness. The reflex, "write a test, run it, green means done" no longer works, and most engineers don't have a replacement reflex ready. You can't assert on exact wording, because the wording changes. You can assert on structure (does the output parse, do the required fields exist, are the counts right), and you can assert on invariants (does it violate a rule it was told to follow), but both of those are weaker than what engineers are used to, and both let real unexpected output through.

Instead of asking whether one output equals one expected answer, you begin asking whether the system satisfies a rubric across a distribution of inputs and repeated runs. That is closer to evaluation than ordinary unit testing, and the result is a confidence level rather than a proof of correctness.

This brings us to...

Failure is silent and graded, not binary

In ordinary software, a failure announces itself. There's an exception, a stack trace, a non-zero exit code, a red log line. Even when a bug is subtle, the system eventually produces a signal — a wrong number, a crash, a timeout. You know something is wrong because the program told you, or because it stopped.

Model output does not crash. A language model that is 95% correct and 5% broken produces a response that looks completely fine. There is no error symbol. There is no exception. The 5% that's wrong is woven into the 95% that's right, in the same prose, in the same format, with the same confidence. The model does not know it's wrong; it has no separate channel for "I am unsure," and even when you give it one, it uses it inconsistently. So the failure sits there, inside an apparently clean document, and nothing in your pipeline raises an alarm.

This means the most dangerous outputs are the ones that almost pass. A response that's grossly wrong is easy to catch (your validators reject it, or a human glances at it and winces). A response that's subtly wrong passes every shallow check and reaches the user. And "subtly wrong" with a language model is not a small category. It includes the model drifting off a constraint because a later instruction indirectly contradicted an earlier one, the model inventing a plausible-sounding value where it should have admitted it didn't know, and the model following the letter of a rule while breaking the spirit. None of these crash. All of them ship.

So you end up building defences against a class of failure that has no signal. You can't catch what you can't see, which means a large fraction of harness engineering is making the invisible visible: adding checks that surface graded drift, adding assertions that flag "this is structurally fine but semantically off," and adding humans in the loop at the points where confidence is low, all of which come with their own tradeoffs.

You're debugging prose, not code

Detecting a bad output is only half the problem.

When the harness breaks, the "code" that caused the failure is often a paragraph of English. The only debugger is your judgment, reading a thousand words and trying to determine what gave the model permission to do what it just did.

And here's the cruelty of it: a single word can be the bug. I have spent hours tracking a regression to one adjective in a prompt, a word that seemed innocuous when I wrote it, that the model was interpreting as permission to do something I never intended. In source code, a one-character bug is a cliché, but it's findable. In a prompt, a one-word bug is invisible. The word doesn't break anything mechanically. It just shifts the model's behaviour a few degrees, and a few degrees is enough to quietly corrupt output before anyone notices.

Worse, your prompt might have a specific order, but it does not execute like a program with explicit control flow. Earlier and later instructions can influence one another in ways that are difficult to isolate.

You can't reason about it the way you reason about a function. You reason about it the way a writer reasons about a paragraph, by feel, by reading it aloud, by asking "What is this actually telling the reader to do?" And then you test your reading by running it and squinting at the output. That is a genuinely different skill from debugging, and most engineers have not had to develop it.

The additive instinct is a trap

Here is the most reliable failure pattern in harness engineering, and the one I fell into the most times before I learned to see it: something breaks, and the instinct is to add a rule to the prompt. The model did the wrong thing? Tell it not to. The model forgot a constraint? Repeat the constraint louder. The model invented something? Add a line that says "do not invent." Same with the classic "MAKE NO MISTAKES!" Each addition feels responsible, even careful. You are, after all, responding to a real failure.

The prompt grows... and grows... and keeps on growing. But the prompt growing is the disease, not the cure.

Every rule you add narrows the space of outputs the model is allowed to produce. That's the point! But there's a major issue. The rules are written in natural language, and natural language rules interact with the models in ways you and I cannot fully predict. A rule that fixes one failure contradicts a rule that fixed an earlier one. A rule that says "be concise" fights a rule that says "include all of these details." A rule that says "never do X" makes the model think about X, and now X appears in outputs where it never appeared before. You don't get a contradiction error. You get a model that is now over-constrained, pulled in enough directions that it can no longer satisfy all of them at once, so it picks one to violate quietly, and you've traded one visible bug for a different, subtler bug.

I have watched a prompt grow from twenty lines to two hundred, with each line added in good faith to fix a real failure, until the prompt was so heavy that it could barely produce anything good anymore. The solution, every time, was to subtract. Yes, to remove rules, to collapse duplicates, to delete the instructions that were subtly fighting each other, to take a constraint out of prose and enforce it in code where it couldn't be contradicted. The jumps in quality came from deletion, not addition. But deletion is the move that feels wrong in the moment, because you are "removing a safeguard," and nothing in your engineering training rewards you for throwing away a rule that once fixed a bug.

Examples steer harder than rules

There's a corollary that makes the additive trap worse: the model learns from examples far more powerfully than it learns from instructions. If you write a rule that says "don't do X" and then include an example that happens to do X, the model will do X. The example wins. You can write the rule in caps, you can repeat it, you can beg, but the example is a demonstration, and a demonstration is a stronger signal than a prohibition.

The deeper lesson is that everything in a prompt is a behavioural program: instructions, examples, the order of the sections, the words you chose, (maybe) even the formatting. There is no "neutral" part of a prompt. Every token pushes the model somewhere. Which means maintaining a prompt is not like maintaining a config file; it's like maintaining a program where every line is executable, with a range of outputs.

The foundation rewrites itself

Most software is built on foundations that hold still. Your language version bumps occasionally. Your framework releases breaking changes every year or two, with a migration guide and a deprecation cycle. The contract between your code and the platform is, for the most part, stable on the timescale of a sprint.

The model is not a stable foundation. The vendor updates it (sometimes without announcement) and the update changes behaviour. A prompt that produced clean output last quarter produces subtly different output this quarter. Not because you changed anything. Because the statistical surface shifted under you. A phrase the old model read as a hard constraint, the new model reads as a suggestion. A format the old model produced reliably, the new model produces with occasional gaps. Your harness was tuned to the old model's failure modes, and the new model has different failure modes, and your defences now guard against problems that no longer occur while leaving the new problems unguarded.

This has become especially evident with the release of Opus 5. Many people are talking about how their previously reliable skills files are just not working with the new model.

This is the part that makes harness engineering fundamentally different from ordinary engineering. You are not building on a platform; you are building on a probabilistic system that rewrites its own behaviour on someone else's schedule. Every contract you write is with a moving target. Your tests can be green and your product can be regressing, because your tests assert against the old model's behaviour and the model moved. There is no "lock the version" escape hatch that holds forever; vendors deprecate old versions, prices change, and eventually you are forced onto the new surface whether you're ready or not.

The practical consequence is that a harness is never "done." It is in a state of continuous adaptation to a foundation that shifts beneath it. You will spend time you didn't budget re-tuning prompts against a model you didn't necessarily choose to upgrade. You will ship a fix for a regression you didn't cause. This is not a failure of planning; it is a structural fact of the substrate, and the only honest response is to build harnesses that are robust to it — minimal prompts that depend on as few model-specific quirks as possible, validation that catches drift, and the humility to expect the next update to break something you can't yet name.

Feedback loops are slow and expensive

A normal edit-compile-test cycle takes seconds. You change a line, you run the test, you see the result, you change it again. The tightness of that loop is what makes engineering bearable; it's what lets you iterate your way to a correct solution without holding the whole system in your head.

A harness feedback loop is nothing like that. One run of a model call costs money, maybe not much, but enough that you can't run it a thousand times the way you run a unit test. If your pipeline has multiple model calls, a single end-to-end test can take from several minutes to several hours.

So you iterate slower. You hold more state in your head. You can't brute-force a solution by trying twenty variations in a minute; you try three, wait, read the outputs carefully, form a hypothesis, try one more. The whole shape of problem-solving changes. Engineers who are used to tight loops start to feel like they're working in slow motion, and the temptation is to make changes blindly, to edit the prompt, push it, and check back later, which is exactly how regressions get introduced, because you stopped observing before you stopped editing.

The cost has a second, subtler effect: it makes you lazy about verification. When each check is expensive, you start checking less. You convince yourself the change is small enough to skip the full run. It is almost never small enough. The expensive loop trains you to under-test, and under-testing with a probabilistic system is how you ship drift.

Nobody can see the work

The last pain is social, and it might be the most draining. "Prompt engineering" is a phrase that makes the work sound trivial. It implies the job is writing prompts, a task that, to anyone who hasn't done it, seems about as hard as writing an email. The months of tightening, the removed contradictions, the silent failures caught before they shipped, all of that is invisible. It looks like you barely did anything.

There's no clean fix for this. The best you can do is make the invisible visible on purpose: track the failure modes you caught, measure the drift you prevented, write down the regressions that didn't ship, and accept that a portion of the value you create will never be legible to the people around you. That's a tax the work carries, and it's worth naming so you don't internalize the false message that you aren't doing much.

The difficulty is the point

Here's the thing I kept circling back to while I was living in this. Every one of these pain points is structurally hard. They aren't hard because the tools are immature or because we haven't figured out best practices yet. They're hard because they come from the nature of the substrate, a probabilistic system with no stack trace, no determinism, no stable contract, and no separation between "the program" and "the data it was trained on." You can't engineer those facts away. You can only build a harness that absorbs them.

And that, honestly, is the moat. If harness engineering were easy, the model would be the product, and anyone with an API key could compete with you. The reason a real application is hard to build on a language model is the same reason it's hard to copy: the work of turning a probabilistic output into a shippable, reliable, tested thing is invisible, painful, and not transferable by reading your prompt.

The pain is the work, and the work is the moat.