You debated the model for weeks.
You compared benchmarks, context windows, cost per token, latency, and coding scores. Then you chose the harness in an afternoon—usually whatever the first tutorial happened to use.
That is backwards.
The model supplies capability. The harness decides what reaches the model, what actions it can take, how tool results return, what state survives the next turn, when the loop stops, and what happens when execution fails. Change those decisions and the same model becomes a different agent.
The right unit of evaluation is not the model. It is the model–harness pair.
The same model can produce a different agent
In April 2026, Endor Labs ran GPT-5.5 through two coding harnesses in its Agent Security League. Through Codex, the model scored 61.5% on functional correctness. Through Cursor, it scored 87.2%—a swing of 25.7 points without changing the model.
20.1% Codex
→23.5% Cursor
One benchmark, not a universal ranking. The controlled variable is the useful part: the model stayed fixed.
Security correctness moved less, from 20.1% to 23.5%. This is one benchmark, not proof that Cursor is always better than Codex. What matters is simple: the model stayed the same; only the harness changed.
Harness-Bench, a 2026 preprint, found the same pattern across 5,194 runs. Its recommendation is equally simple: score the model and harness together.
The loop is only the center of the system
“Harness” is often used as a loose synonym for the while-loop around an LLM: call the model, execute a tool, append the result, repeat. That loop is only the center of the system.
The loop itself is simple: the model proposes an action, the harness executes it, and the result returns. Production complexity lives around that loop—in context, state, permissions, stopping rules, recovery, and verification.
A production harness does six connected jobs:
Observe
Observation interfaceterminal · DOM · diffs · tool resultsAssemble
Context managerinstructions · memory · compactionReason
Modelproposes the next actionAct
Tool interfaceshell · browser · APIs · MCPObservation and context shape the evidence. Control chooses what happens next. Action exposes tools, state preserves continuity, and verification decides what may run and what counts as done. The model sees only the world these layers show it.
That is why harness changes can look like intelligence changes. A clear diff improves code review. A compact tool result improves planning. A test loop enables self-correction. The model may be identical in every case.
The most important harness code is often the translation layer on either side of the model. Before a call, it chooses between a full file, a retrieved excerpt, a summary, a diff, or nothing. After a tool call, it returns raw output, a typed result, an error category, or a truncated blob.
Those choices change the next decision. A successful tool can return output too noisy for the model to notice the answer. A context compactor can preserve the plan while deleting the reason the previous attempt failed.
This is how a capable model can drift away from reality: it reasons with stale context, incomplete tool output, or the wrong workspace state. Good harnesses do not maximize context. They preserve the evidence needed for the next decision.
Those defaults are part of the product. A model–tool loop can fit on one screen. What matters is not how short the integration looks, but where its behavior lives when a task fails, resumes, or asks to cross a permission boundary.
Evaluate the pair, then diagnose the layer
Model leaderboards encourage the wrong debugging sequence. When an agent fails, teams swap the model before asking what the model saw, what its tools returned, or why the loop stopped.
My minimum eval contract has four parts: a target, a versioned dataset, a metric, and a threshold. For an agent, the target should be the complete model–harness pair, not the model endpoint alone.
Evaluate a model–harness pair under a fixed task, environment, budget, and scoring rule. Then diagnose at the transition level.
- 01ObservationEvidence missing
- 02ContextEvidence lost
- 03ModelInvalid action
- 04ToolWrong execution
- 05ResultResult misread
- 06VerificationFalse completion
The model proposes the next action. It does not decide what evidence arrives, what tools are permitted, what state survives, or what counts as complete.
Pick the harness as deliberately as the model.