What Is an AI Harness and Why It Matters More Than the Model

For months I have been writing on this blog about separate pieces: how I cut my token usage by 80%, how I set up agent teams, how I let agents write production code with SDD. All of those pieces share a name: harness engineering. The concept is not new. The idea that what surrounds the model matters as much as the model itself, or more, has been around for a while among people who build and evaluate agents. But in the last few weeks it has gone from hallway jargon to a term that shows up, almost at the same time, on the blogs of OpenAI, Anthropic, LangChain and MongoDB.

If you have seen it go by and you are not sure what it is, or why suddenly everyone is talking about it, this article is for you.

The equation that sums it all up

Let’s start with the only formula you need to remember:

Agent = Model + Harness

The model is the LLM: Claude, GPT, Gemini. The harness is everything else: the code, the configuration and the execution logic that wrap the model and are not the model. LangChain puts it in a line that has become a motto: “if you’re not the model, you’re the harness”.

The name is a deliberate metaphor. A harness is the set of reins, saddle and bit you use to control a horse. You have an animal with enormous power but no direction; the harness does not make it stronger, it makes it governable. That is exactly what the harness does to an LLM: it does not make it smarter, it makes it reliable.

And here is the twist that made the term go viral: the bottleneck is no longer the model’s intelligence, it is the quality of the harness.

Why this matters now (and not a year ago)

There is concrete evidence of how much the harness weighs. A Stanford team measured it in the Meta-Harness paper: optimizing only the harness, without touching the model, gains 7.7 points of performance with 4 times fewer tokens. Same brain, different wrapper, different result. And that is in the lab; the production cases from Vercel and LangChain you will see below are even more striking.

MongoDB takes it a step further with a comparison that stings. Remember Sculley’s famous diagram of machine learning systems in production? The one where the “ML code” box was a tiny square surrounded by an ocean of infrastructure: data collection, serving, monitoring, resource management. Agents are the same. The interaction with the LLM is barely 1-2% of the real engineering work. The other 98% is harness.

Through 2024 and a good part of 2025 the message was “wait for the next model”. Each new frontier model covered the gaps of the previous one. But models have settled into a very high and very similar range of capability, and suddenly the interesting question is no longer which model you use but what you have built around it. That is where harness engineering was born as a discipline with its own name.

Thinking this way has a lovely side effect. If your value is in the harness and not in the model, the model becomes an interchangeable part. As BettaTech describes it in his video on the topic (in Spanish): you have a system where you simply swap the brain. A better or cheaper model comes out and you plug it in without rewriting anything. Your investment, the harness, stays.

The anatomy of a harness

What exactly is inside that 98%? MongoDB splits it into six layers, and it is the best way of organizing it I have seen:

1. State and persistence. It stores the execution context, the checkpoints and the intermediate outputs. It lets you resume from where things broke instead of starting from scratch.
2. Security and governance. Who the agent is when it acts and what it is allowed to do. When it calls a tool or an API, it has to do it with the identity and permissions of the right user (not with full access to everything), stick to the minimum it needs for its task, and leave a record of every action so it can be reviewed later. This is the layer where most enterprise projects get stuck before reaching production.
3. Orchestration and tool use. The agent loop: perceive → plan → act → call a tool. Coordination, rate limiting, sandboxed execution.
4. Memory. Working memory (the current turn) and persistent memory (a short-term cache plus long-term stores: episodic, conversational, entity memory).
5. Observability. It records decisions, reasoning traces and tool calls. It is not the same as monitoring: observability asks “was the decision right?”, not “did it respond?”.
6. Evals. The tests that measure whether the agent does its job well. They do not run live on every request: they run separately, on what observability recorded, and they are what tell you whether the agent is good enough to go to production.

You do not need to build all six on day one. But it is worth knowing they exist, because most people start and finish at layer 3 and then wonder why their agent “sometimes fails at the simplest things”.

Guides and sensors: Martin Fowler’s framework

For the developer who uses coding agents every day, which is where I live, the most useful framework is not the six layers but Martin Fowler’s. He splits the harness into two kinds of control:

Guides (feedforward). Controls that act before the agent does anything. They anticipate its behavior and steer it. They are your CLAUDE.md and AGENTS.md, the code conventions, the architecture docs, the instructions in your skills. You tell the horse where to go before it starts moving.

Sensors (feedback). Controls that act after the agent acts, and let it correct itself. Linters, tests, type checks, a headless browser with Playwright that checks the UI works. The agent acts, the sensor detects the problem, the agent fixes it.

Sensors are especially powerful when they emit signals designed to be consumed by an LLM. A linter message that does not just say “error on line 42” but “error on line 42, this is how you fix it” closes the self-correction loop without you stepping in.

Fowler adds an important nuance about cost. Running these controls can be computational (deterministic, fast, cheap: a linter, a unit test) or inferential (semantic, slow, expensive: another LLM reviewing the code). The practical advice is to spread them over time: cheap checks before integrating, expensive ones afterwards, in the pipeline. Do not put a reviewer agent to read every line when a type check tells you the same thing for free.

The counterintuitive lesson: less is more

This is the part almost everyone gets wrong at first, me included.

Intuition says: if the harness is what matters, I will build a huge harness. I give the agent fifty hyper-specialized tools, plug in three MCP servers, write a 600-line CLAUDE.md, constrain its reasoning step by step. A harness on steroids, an unstoppable agent.

The exact opposite happens. The more complex you make the harness, the worse it works.

The example everyone quotes comes from Vercel. They have an agent called D0, v0’s cousin, for querying and analyzing large amounts of data. The first version followed intuition: specialized tools to write SQL, to connect to each database, wrappers that guided the model’s reasoning with surgical precision. The idea was to make the model’s job easier.

Then they tried the opposite: taking the training wheels off. They removed 80% of the tools and left only good old Unix utilities (grep to search files, cat to read them, ls to list directories) and let the model figure things out.

The result: the success rate went from 80% to 100%, token usage dropped 37%, and average latency fell 3.5 times. The worst case says it best: a query that used to spin for 724 seconds and end up failing was solved by the new version in 141. Same model capability. (The evaluation was only five queries, so it is a strong signal, not a law.)

LangChain confirms it from another angle: keeping the model fixed and touching only the harness, it raised its coding agent 13.7 points on Terminal Bench 2.0 (from 52.8% to 66.5%) and took it from the bottom of the table to the top 5. And the law firm Harvey, applying harness engineering to real legal tasks, raised its agents’ accuracy from 0.853 to 0.913 on its own benchmark, enough to beat frontier models.

Why does it work this way? Because every tool you add is one more decision the model has to make, one more description taking up context, one more path to get wrong. A frontier model knows how to use grep. What it does not know is how to pick correctly among your forty custom tools with similar names. Harness complexity is debt, not an asset.

This ties directly into everything I wrote about token optimization (in Spanish): every tool has a narrow optimal window, and outside of it it costs more than it gives.

The other enemy: context that rots

There is a second problem that anyone who has used Claude Code in long sessions has suffered: the longer you go, the dumber the agent gets.

And it is not your imagination, it has been measured. Even though context windows are huge today (200K, a million tokens), results start degrading long before the window fills up. Chroma’s research on context rot and the classic Lost in the Middle paper document it: the model starts losing quality and forgetting what sits in the middle well before the limit. The specific thresholds that get passed around (that it starts at 20% usage, that you should clear at 40%) are rules of thumb, not laws: degradation depends more on the absolute number of tokens and on the type of task than on a fixed percentage. The space is there, but the quality is not.

That is why context management is a central piece of the harness, and why memory is not optional. The idea is easy to picture: the context window is a small box inside the model, and your job is to move things out (to a file system, to a database) so that box is never full. The agent looks up external memory when it needs it instead of always dragging it along.

This is the filesystem-as-memory pattern, and it is exactly what I use in my vault with the .agent/ directory: a memory.md that works as an index, completed tasks archived by date, reusable knowledge in separate files. The agent does not load everything: it follows the links it needs. And you do not have to build it by hand: there is a growing ecosystem of memory tools for agents (Engram, claudemem and others) that package this pattern and manage it for you. Anthropic has published specific guides on how to build harnesses for long coding sessions, and almost all of them revolve around the same thing: a structured JSON file with the pending tasks, which agents pull from, and subagents that start with an empty context instead of inheriting the parent’s.

That last point is key. When you launch a subagent, what you do not want is for it to inherit the 80K tokens of context the orchestrator already has. You want it to start clean, with only the information for the specific task it has to do. Less context, more efficient agent. It is the basis of the agent teams I set up.

Verification: make it prove it, not say it

Now we reach the most dangerous problem of all, and the one the harness has to solve no matter what.

You cannot trust what the AI tells you. Code generated by an LLM can look 100% convincing and have a subtle bug buried where it does not meet its goal. Remember what the model is trained for: to sound plausible, to give you answers that look right. Looking right does not mean being right.

So the design question is: how do you make the agent not just say “done”, but prove it?

The answer is to give it verification tools and require it to use them. Automated tests, written by you or by it, but that actually pass. A headless browser that opens the app and tries the flow. Chrome DevTools. Never, ever let the only source of truth about whether something works be the same agent that built it. It has to prove it, and software is lucky enough to be verifiable: take advantage of that.

Since the agent generates more and more code and reviewing all of it by hand does not scale, an extra layer is taking shape: using AI as a reviewer of AI. Adding to the harness explicit instructions on how to validate and review the work, and agents dedicated only to that. It is Fowler’s sensor layer taken to the extreme: a reviewer agent inside the pipeline itself, like the one I set up in my 9-agent pipeline (in Spanish).

What a real harness looks like

Let’s get practical. A project with a decent harness today, with Claude Code, looks something like this:

my-project/
├── .claude/
│   └── agents/
│       ├── implementer.md     # writes code
│       ├── lead.md            # orchestrates
│       └── reviewer.md        # verifies
├── CLAUDE.md                  # entry point: shared rules
├── docs/
│   ├── architecture.md
│   └── conventions.md
├── .agent/
│   ├── memory.md              # persistent memory index
│   └── tasks/                 # structured tasks
└── src/

Three ideas to take from that skeleton:

The harness lives in the repository. It is not configuration you keep on your machine: it is part of the project, versioned in git. This has a nice consequence: the harness is self-editable. If you notice an agent is not doing its job well, you change its prompt or its rules in the repo and next time it works better. The harness evolves with the project.

The protocol is mandatory. Every agent, to be part of the harness, follows the same rules: the first thing it does is read CLAUDE.md and the architecture and conventions docs. Without a shared protocol there is no team, just loose agents.

Orchestrator and workers. A lead agent that coordinates and spawns, and small subagents that do the work with minimal context. It is the model of Anthropic’s multi-agent research system: the user sends the query, the orchestrator iteratively delegates to subagents that search and save to memory, and at the end the orchestrator returns the result.

My advice, and that of practically everyone who writes about this: start small and grow. Do not build the three agents and the six layers on day one. A good CLAUDE.md, a couple of verification commands the agent is required to run, and a memory file. Understand each piece before adding the next one. The minimal harness you understand beats the complete harness you do not control. It is Vercel’s lesson again.

What this means for you

If I had to boil all of harness engineering down to four takeaways:

• The model is the small piece. Your competitive advantage is in the harness, not in which LLM you use. And because the harness is yours, it survives every model change.
• More harness is not a better harness. Fewer, more general tools almost always win. Complexity is debt.
• Context rots before it fills up. Move state out to external memory and start subagents clean.
• Make it prove it. Without automated verification you do not have an agent, you have something that sounds convincing and little else.

And one last, half-personal observation. When I first read “harness engineering” I thought it was another inflated term to sell hype. But when you look closely, it describes something concrete and real: the skills, the code navigation rules, the memory in .agent/, the agent teams, the mandatory checks. All of that is harness. The difference between treating AI as a chatbot you ask for things and treating it as an engine you build a system on is, literally, this discipline.

The horse always had the power. What was missing were the reins.

---

If you want to go deeper: Martin Fowler’s article on harness engineering is the best source on the guides and sensors framework; OpenAI’s tells the story of a three-person team that spent five months without writing a line by hand; and MongoDB’s has the clearest breakdown of the six layers.

P.S.: If you are building your own harness for agents, or something in here does not convince you, you can find me on Twitter as @lm_martinbar. And if you want the background, this article is the natural follow-up to how I set up agent teams and spec-driven development with OpenSpec.