A Field Guide to Agent Harness Engineering: The Architecture Behind Reliable AI Agents

Why I am writing this

For most of the last three years, the implicit thesis of agentic AI has been simple: a better model produces a better agent. Buy the frontier model, write a good prompt, wire up some tools, and reliability follows.

Between March and June 2026, a cluster of papers landed that, read together, argue something close to the opposite. The binding constraint on real-world agent reliability is increasingly not the model. It is the harness  the runtime system that wraps the model and governs how it observes, acts, remembers, recovers, and is verified.

This is not a vocabulary game. It changes how you staff a team, where you spend compute budget, what you put in a benchmark, and what you can honestly promise a board. Below I work through seven papers, group them into a coherent picture, and end with what I think it means in practice. References are listed in full at the end so the claims here are traceable to their sources rather than to me.

A note on framing for my PhD students: I have deliberately kept the formal objects (the labeled-transition-system semantics, the component algebra) close to how the authors state them, because the precision is the point. For my consulting readers, each section closes with a “so what” that you can act on without the math.

1. The shift in one sentence

Across all seven papers, one claim recurs in different vocabularies:

Agent capability is increasingly externalized out of model weights and into the surrounding runtime and the engineering of that runtime, the harness, is now a first-class determinant of system behavior.

The Externalization review (Zhou et al., 2604.08224) gives the cleanest theoretical framing. Drawing on Donald Norman’s idea of cognitive artifacts, it argues that agent infrastructure matters not because it bolts on extra parts, but because it transforms a hard cognitive burden into an easier one the way a shopping list turns “recall the items” into “recognize the items,” or a map turns hidden spatial relationships into visible structure. In their taxonomy: memory externalizes state across time, skills externalize procedural expertise, protocols externalize interaction structure, and the harness is the unification layer that coordinates all of them into governed execution.

That single reframing capability lives in the runtime, not only in the weights is the thread connecting everything that follows.

2. What exactly is a “harness”? Two attempts at a definition

The field has used “harness,” “scaffold,” “runtime,” and “agent OS” interchangeably for a while. Two of these papers try to fix that.

2.1 The six-component formalism: H = (E, T, C, S, L, V)
The most rigorous treatment is the survey by Meng, Wang et al. (202604.0428), Agent Harness for Large Language Model Agents. It defines an agent harness as a software system implementing six runtime governance functions:

• E — Execution loop. Manages the observe–think–act cycle: turn sequencing, termination conditions, error recovery.
• T — Tool registry. A typed, validated catalog of tool interfaces; routes and monitors invocations.
• C — Context manager. Governs what enters the model’s context window each turn — compaction, retrieval, prioritization.
• S — State store. Persists task state across turns and sessions; enables recovery from partial failure.
• L — Lifecycle hooks. Pre/post-invocation interception for authentication, logging, policy enforcement, instrumentation.
• V — Evaluation interface. Structured trajectory capture in standardized schemas that external benchmark frameworks can consume.

The discipline of this definition is that it draws boundaries. The authors map each component to a corresponding production failure mode — execution runaway (E), tool misuse (T), context blowout (C), state loss on failure (S), unmonitored side effects (L), and unobservable behavior (V). They then argue E and T are necessary (without E there is no multi-step execution to govern; without T the agent cannot act), while a full-stack harness implements all six with production-grade reliability.

What makes the formalism more than a checklist is the labeled-transition-system (LTS) treatment of the execution loop. The authors model E as a transition system over states like idle, observing, invoking-model, dispatching-tool, awaiting-tool-result, committing-state, terminated, and surface three correctness properties that informal descriptions cannot express:

• Safety — the system never enters a state from which termination is unreachable (no runaway).
• Liveness — from every reachable non-terminal state, a terminal state is reachable.
• Determinism — for reproducibility, environment non-determinism must be isolated at tool-call boundaries.

This is where the definition earns its keep. Using the LTS lens, they classify systems cleanly:

• ReAct is not a harness — a bare while-loop with no error-recovery arc, so its transition function is partial and safety fails. It is a framework primitive.
• AutoGPT is a (monolithic) harness — its transition function is total over the documented events, including error recovery; its reliability problems come from weak implementation guarantees (non-idempotent writes, non-atomic commits), not a missing structure.
• LangGraph is a topology-encoded harness — a DAG whose acyclicity guarantees liveness by construction, but whose context management (C) is implicit in graph topology rather than an active runtime policy.

The same paper assembles a completeness matrix over 22 representative systems and reports an uncomfortable empirical regularity: the V (evaluation) and L (lifecycle) columns are the most systematically under-implemented across the ecosystem — V is partial or absent in 14 of 22 systems. In other words, the two components that give you accountability and the ability to measure are exactly the ones teams skip.

2.2 The Control–Agency–Runtime decomposition

He, Zhou et al. (202603.1756), Harness Engineering for Language Agents, approach the definition from a complementary angle, decomposing the harness layer into Control, Agency, and Runtime (CAR): what instructions remain authoritative, what actions are available, and how state is carried forward and how failures are handled over time.

Two of their contributions are directly useful to practitioners and worth dwelling on:

1. They audit 63 harness-relevant works and find a visibility gap between academic papers and public engineering notes — much of the real knowledge lives in vendor blogs, not citable literature.
2. They argue that many reported “model” gains may be partly harness-sensitive rather than purely model-driven, and propose HarnessCard — a lightweight disclosure artifact for reporting the harness configuration alongside any agent result.

That second point should make every researcher slightly uncomfortable. If a paper reports that Model B beats Model A on an agentic benchmark but does not disclose the harness, the comparison may be confounded. CAR and HarnessCard are an attempt to make the harness a reported variable rather than an invisible one.

They also pre-empt the obvious objection — “isn’t this just software engineering?” — by noting that harness control artifacts are often linguistic objects: instructions, repository maps, tool descriptions, summaries, approval prompts, grader criteria, progress files. Governance here is mediated through language, not only through low-level system calls. That is what makes it a distinct object of study.

So what (for leaders): Before you compare two agent vendors or two internal builds, demand a harness disclosure. “Which model” is the wrong first question. “Which harness, and how complete is it across E/T/C/S/L/V” is the right one.

---

3. The evidence that the harness is the binding constraint

The strongest part of the Meng/Hu survey is that it does not merely assert the thesis — it marshals convergent evidence that harness-level changes, with the model held fixed, move the numbers as much as or more than model upgrades. The pattern, drawn from the survey’s account of independent studies and deployments:

• A change to only the edit-tool format — no model change — reportedly produced an order-of-magnitude jump on a coding benchmark for certain models (on the order of 6.7% → 68.3% for one model). Treat the exact figure as a practitioner report, but the direction and magnitude are the signal.
• LangChain’s DeepAgents improved on TerminalBench 2.0 from 52.8% to 66.5% (+26%) through harness-layer changes alone — self-verification prompts, middleware-based context injection, lifecycle hooks — with zero model changes, moving from outside the top 30 to the top 5.
• Meta-Harness (an automated harness-optimization framework) reached 76.4% on TerminalBench-2, surpassing hand-engineered approaches (74.7%), and delivered gains on held-out mathematics and text classification without model changes.
• HAL (Holistic Agent Leaderboard), a standardized evaluation harness running ~21,730 rollouts across nine models and nine benchmarks, cut evaluation time from weeks to hours and eliminated common implementation bugs — implying a non-trivial fraction of what prior literature logged as agent failures were actually harness failures.
• AgencyBench found that proprietary models perform best within their native execution ecosystems — direct evidence of harness–model coupling, a phenomenon a capability-only study cannot even see because it holds the environment constant or ignores it.

The survey is careful — and so am I — to separate peer-reviewed evidence (HAL, SWE-bench, OSWorld) from preprint and practitioner reports (AgencyBench, SkillsBench, SandboxEscapeBench, vendor accounts). The honest summary: no single number is load-bearing, but the convergence across independent sources is hard to dismiss.

There is also an economic edge to this that lands directly in a CAIO’s lap. The survey ties harness design to cost, not just accuracy. When a benchmark task consumes ~1 million tokens per execution, that load is driven by the harness’s context-management policy — what gets injected each turn, how memory is retrieved, how long the loop runs — not by model verbosity. The emerging operating metric is Cost Per Task (CPT) rather than tokens-per-second. As they put it: the design of the harness is the design of the cost curve.

So what (for leaders): If your agent program’s unit economics are bad, the fix is unlikely to be a bigger model. It is almost certainly a context-management and loop-termination problem in the harness. Instrument CPT per task type before you renegotiate your model contract.

---

4. The three engineering eras

The Meng/Hu survey frames a clean historical arc that I find pedagogically useful, and which I now use with clients to locate where their org actually is:

1. Prompt engineering (2022–2024): optimize the input text. “What text gets better outputs?” Chain-of-thought, few-shot, instruction tuning.
2. Context engineering (2025): optimize what the model sees. “What structured information do we assemble each turn?” Memory retrieval, tool-result formatting, context-window management.
3. Harness engineering (2026): optimize the full governance infrastructure. “What controls, feedback loops, and execution constraints make the system reliable?” All six components as an integrated whole.

Each era is a strict expansion of scope, not a replacement. Most enterprises I advise are operating at era one or two while their problems are squarely era three. That mismatch is the single most common root cause of stalled agent pilots I see.

The survey also traces the deeper lineage — the harness synthesizes three older traditions: the governance-wrapper pattern from software test harnesses (JUnit/xUnit), the interface-standard pattern from RL environments (OpenAI Gym’s reset()/step()/observe()), and the failure-mode catalog from early LLM agents (AutoGPT, BabyAGI). None alone is sufficient; the harness is what you get when you combine the wrapper, the contract, and the hard-won list of what goes wrong.

---

5. How to represent a harness: code vs. natural language

If the harness is now a first-class object, the next question is how to express it. Two papers give opposite-but-complementary answers.

5.1 Code as the substrate (Ning, Tieu et al., 2605.18747)

Code as Agent Harness argues that code is no longer just the agent’s output — it is the operational substrate for reasoning, acting, environment modeling, and execution-based verification. The survey organizes this across three layers: the harness interface (code connecting agents to reasoning/action/environment), harness mechanisms (planning, memory, tool use, plus feedback-driven control), and scaling from single- to multi-agent settings where shared code artifacts enable coordination, review, and verification.

5.2 Natural language as the substrate (Pan et al., 2603.25723)

Natural-Language Agent Harnesses takes the inverse position. It observes that harness logic is usually buried in tightly coupled controller code, which makes it hard to inspect, compare, transfer, and ablate. So they externalize the high-level control logic into NLAHs — editable natural-language documents describing run-level harness policy — executed by a shared Intelligent Harness Runtime (IHR) that interprets them into agent calls, handoffs, state updates, validation gates, and artifact contracts.

The empirical result matters: across coding, terminal-use, and computer-use benchmarks, IHR-executed NLAHs achieve outcomes comparable to code and prompted realizations, while exposing a much shorter, inspectable surface — and module ablations show the explicit harness modules are individually analyzable. Their primitives (roles, contracts, durable artifacts, verification gates, delegation boundaries) are essentially the CAR/six-component ideas rendered as a readable spec a compliance officer or domain expert could edit.

They also answer the obvious worry — won’t stronger models make the harness unnecessary? — by arguing that natural-language control is durable: as tasks span many context windows, robust progress depends less on one-shot phrasing and more on durable state surfaces, validation gates, and clear responsibility boundaries.

The synthesis: code gives you executability and verification; natural language gives you inspectability and portability. The mature stack will use both — natural-language policy documents (think AGENTS.md, skill bundles, CLAUDE.md) compiled or interpreted onto a code-grounded runtime. This is, not coincidentally, exactly how the leading coding-agent harnesses are already built.

6. How to improve a harness: the optimization frontier

The newest papers stop treating the harness as something humans hand-tune and start treating it as something you can search and co-evolve.

6.1 HarnessForge — joint harness–policy evolution (Chen et al., 2606.01779)

HarnessForge makes the sharpest conceptual move of the set. It formulates an agent system as a harness–policy pair and argues that prior work optimized one or the other but rarely the compatibility between them. It defines an explicit adaptation space separating harness-level execution structure from policy-level reasoning behavior, then co-evolves the two via fault-guided harness tailoring (let observed failures drive harness edits) and harness-conditioned policy alignment (align the reasoner to the harness it will actually run in).

This is the natural endgame of the harness–model coupling result from §3: if model and harness are coupled, you should optimize them jointly, not pretend they are independent.

6.2 Observability-driven automatic evolution (AHE, 2604.25850)

A concurrent line — Agentic Harness Engineering — automates the manual “inspect-trajectories-and-revise” loop through three observability pillars: component observability (every editable harness component gets a file-level, git-tracked, revertible representation), experience observability (millions of raw trajectory tokens distilled into a layered, drill-down evidence corpus), and decision observability (every edit paired with a self-declared prediction, falsified against the next round’s outcomes — “every edit becomes a falsifiable contract”). The reported result: ten iterations lifting pass@1 on Terminal-Bench 2 from 69.7% to 77.0% with the base model frozen, and the evolved harness transferring to SWE-bench-Verified — evidence the gains encode general engineering experience, not benchmark-specific tuning. (I include this as a closely related work; it is not in your original list but it is the empirical companion to HarnessForge and worth your students’ time.)

6.3 A-Harness — verification-gated skill orchestration (Huang et al., 2605.00663)

Affordance Agent Harness applies the harness idea to embodied / robotics affordance grounding, where an agent must decide where and how to interact in messy open-world scenes. Prior systems chained skills (detection, segmentation, zoom-in, web search, interaction-imagination) along a fixed pipeline poorly matched to per-instance difficulty. A-Harness replaces the fixed script with a closed-loop, budgeted runtime: a shared interface over heterogeneous skills, an evidence store with cost control, episodic memory providing priors for recurring object categories, and a Router that adaptively selects and parameterizes skills with verification-driven retries.

Two things to notice. First, it recasts the task as “budgeted evidence seeking” rather than single-pass prediction — a profoundly harness-shaped reframing. Second, the components map almost one-to-one onto the six-component formalism (router ≈ E, skill interface ≈ T, evidence store/memory ≈ C+S, verification gates ≈ V, cost control ≈ L). The same architecture that governs a coding agent governs a robot’s perception loop. That convergence is the strongest sign that “harness” is a real, domain-independent abstraction and not a coding-agent fad.

7. Putting it together: one picture

Here is the synthesis I now draw on a whiteboard for clients.

Layer 0 — Model (weights). Necessary, not sufficient. Capability ceiling.

Layer 1 — Externalized capability (Zhou et al.). Memory (state across time), skills (procedural expertise), protocols like MCP/A2A (interaction structure). These pull cognitive burden out of the weights.

Layer 2 — The harness (Meng/Hu; He/Zhou). The unification layer that coordinates Layer 1 into governed execution. Formally H = (E, T, C, S, L, V), or CAR if you prefer the coarser cut. This is where reliability, cost, and safety are actually determined.

Layer 3 — Representation (Pan et al.; Ning et al.). How the harness is expressed — code for executability/verification, natural language for inspectability/portability. Best practice uses both.

Layer 4 — Optimization (Chen et al.; AHE). The harness is no longer hand-tuned; it is searched, co-evolved with the policy, and continuously improved under observability.

Cross-cutting — domain generality (Huang et al.). The same architecture governs coding, computer-use, and embodied agents.

The intellectual move that unlocks all of this is the one from the survey: stop studying the agent’s components in isolation, and start studying the runtime that integrates them as a unified object. The field understood the engine well and the chassis barely at all. These papers are the chassis getting its due.

This five-layer picture is not a slide for me, it is the working checklist my team at AgentsArchitects runs client systems against: which layers exist, which are implicit, and where the next failure is most likely to originate. Most of the stalled agent programs we are brought in to fix turn out to be missing Layer 2 entirely.

8. What this means in practice

For Students:

• The open problems are unusually tractable and unusually impactful right now. Pick one: a formal harness specification language with verifiable safety/liveness over the LTS; a cross-harness benchmark that controls for harness when comparing models; a security taxonomy for harness attack surfaces (container escape, instruction-authority confusion); or regression-free harness improvement (how do you edit a harness without silently breaking a passing case?).
• Whatever you build, report your harness. Adopt HarnessCard-style disclosure now. Your results will age better and your reviewers will trust them more.
• The V-component is the field’s blind spot (under-implemented in 14 of 22 surveyed systems). Evaluation infrastructure is undervalued and therefore high-leverage. Build there.

For Fellow Researchers:

• The harness–model coupling result (AgencyBench) means many published model comparisons are confounded. There is a genuine replication-and-reanalysis agenda here: re-run prior agentic comparisons controlling for harness and see what survives.
• Code-as-substrate and natural-language-as-substrate are not competitors; the interesting work is the compiler between them — turning an inspectable NL policy into a verifiable code runtime without losing either property.

For CAIOs and Engineering Leaders — the part I am asked about most:

1. Reallocate. If your agent budget is ~90% “which model / how much fine-tuning” and ~10% harness, you are inverted relative to where reliability is actually decided. The harness is where I would put the marginal engineer.
2. Hire for harness skills. The role is distinct: someone fluent in execution-loop semantics, context management, state persistence, observability, and sandboxing — not (only) a prompt specialist and not (only) an ML researcher.
3. Measure Cost Per Task, not tokens/sec. Your context-management policy is your cost curve. Most agent ROI problems are harness-economics problems in disguise.
4. Demand harness disclosure from vendors. Before any bake-off, get each vendor’s E/T/C/S/L/V completeness. A model demo without harness disclosure is a magic trick, not a procurement input.
5. Treat security as a harness property. Container escape and instruction-authority confusion are isolation and lifecycle failures (L), not model-alignment failures. They will not be fixed by a better model.
6. Plan for evolving harnesses. The frontier (HarnessForge, AHE) is automated harness improvement under observability. The competitive moat over the next 18 months will be the rate at which your harness improves, not the model you license.

A practical starting point that costs nothing: take the six-component model in §2 and score your own production agents — full, partial, or absent on each of E, T, C, S, L, V. The gaps usually map directly onto the incidents your on-call team is already firefighting. That self-audit is the first engagement we run at AgentsArchitects, we start with your architecture, not a sales deck, and if the honest answer is “your harness is already complete,” that is a perfectly good outcome too.

The one-line takeaway I leave clients with: 2025 was the year of agents; 2026 is the year of agent harnesses. The model is the engine, but you are buying — and your customers are experiencing — the whole vehicle. Build the chassis.

References

The following are the primary sources synthesized in this post. Where I have cited specific quantitative figures, they originate in these works (and, in several cases, in practitioner reports those works themselves cite and explicitly flag as non-peer-reviewed); readers should consult the originals and weight peer-reviewed versus preprint/practitioner evidence accordingly.

1. X. Ning, K. Tieu, D. Fu, T. Wei, … J. He. “Code as Agent Harness.” arXiv:2605.18747 [cs.CL], 18 May 2026. https://arxiv.org/abs/2605.18747
2. L. Pan, L. Zou, S. Guo, J. Ni, H.-T. Zheng. “Natural-Language Agent Harnesses.” arXiv:2603.25723 [cs.CL], 26 Mar 2026 (v2, 18 May 2026). https://arxiv.org/abs/2603.25723
3. C. He, X. Zhou, D. Wang, H. Xu, W. Liu, C. Miao. “Harness Engineering for Language Agents: The Harness Layer as Control, Agency, and Runtime.” Preprints.org, DOI:10.20944/preprints202603.1756 (v1, 23 Mar 2026; v2, 23 Apr 2026). https://www.preprints.org/manuscript/202603.1756
4. Q. Meng, Y. Wang, L. Chen, Q. Wang, C. Lu, W. Wu, Y. Gao, Y. Wu, Y. Hu. “Agent Harness for Large Language Model Agents: A Survey.” Preprints.org, DOI:10.20944/preprints202604.0428 (v1, 07 Apr 2026). https://www.preprints.org/manuscript/202604.0428/v1
5. C. Zhou, H. Chai, W. Chen, Z. Guo, … W. Zhang (21 authors; SJTU, SYSU, Shanghai Innovation Institute, CMU, OPPO). “Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering.” arXiv:2604.08224 [cs.SE], 9 Apr 2026. https://arxiv.org/abs/2604.08224
6. M. Chen, C. Lv, G. Zhang, H. Chang, S. Zhou (Beihang University; Tsinghua University). “HarnessForge: Joint Harness and Policy Evolution for Adaptive Agent Systems.” arXiv:2606.01779 [cs.CL], 1 Jun 2026. https://arxiv.org/abs/2606.01779
7. H. Huang, J. Shi, Y. Li, Y. Chen (HKUST(GZ); Knowin AI; Harbin Engineering University). “Affordance Agent Harness: Verification-Gated Skill Orchestration.” arXiv:2605.00663 [cs.RO], 1 May 2026. https://arxiv.org/abs/2605.00663

Closely related works referenced in passing (not in the original reading list, included for completeness):

1. Agentic Harness Engineering (AHE). “Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses.” arXiv:2604.25850, 28 Apr 2026. https://arxiv.org/abs/2604.25850
2. A. Kapoor et al. “Holistic Agent Leaderboard (HAL): The Missing Infrastructure for AI Agent Evaluation.” arXiv:2510.11977 (accepted ICLR 2026).
3. C. Jimenez et al. “SWE-bench: Can Language Models Resolve Real-World GitHub Issues?” ICLR 2024.