Harness Engineering: What I’d Build Around an AI Agent

Harness engineering is the name this source gives to a shift in AI-agent development: treating the infrastructure around the model as part of the product, rather than spending all the effort on prompts and model selection. I mean the code that assembles context, controls tools, checks results, limits spending, and records execution. This is not an announcement of a new model or API. My takeaway is a production design principle: I would not treat an agent’s ability to produce an answer as evidence that it can safely complete a workflow.

The supplied article, What Is Harness Engineering? The Discipline That Makes AI Agents Reliable, argues that this surrounding infrastructure matters more than model upgrades. I agree with the engineering direction. I am less comfortable treating its headline percentages as general rules for somebody else’s application.

What I mean by an agent harness

I find the distinction easier to understand through responsibilities. A prompt tells the model what I want. Context engineering determines what information it receives at a particular step. The harness controls how the whole execution proceeds, including what happens when something goes wrong.

The source places context engineering inside the broader harness. It also includes tool orchestration, verification, cost controls, and observability. I would use that definition rather than equating a harness with an agent framework. My acceptance criteria would concern behavior under failure, not whether I had adopted a particular library.

This connects directly to how I would approach AI-assisted editorial workflows. Before letting an agent update a newsroom CMS, I would want explicit answers to three questions: which article did it read, which operation is it allowed to perform, and what evidence lets the application accept the result? A better instruction alone would not satisfy those requirements.

The five harness components I would review

The source groups the work into five components. I would turn those into a design-review checklist rather than five separate services:

• Context assembly: I would select information for the current step, not automatically attach every available document, previous response, and tool description.
• Tool orchestration: I would define accepted inputs, usable outputs, failure responses, and timeout behavior for each operation exposed to the agent.
• Verification: I would check results before passing them into another step. The source distinguishes structural checks from semantic review; I would keep that distinction visible.
• Cost envelopes: I would enforce a task budget outside the model’s instructions, with a structured outcome when the remaining allowance cannot support another call.
• Observability and evaluation: I would capture execution traces and repeatedly run representative tasks, so debugging and release decisions have evidence behind them.

I would not give each component equal implementation effort on day one. My starting point would be whichever boundary currently lets an invalid result continue unnoticed. That follows the source’s recommendation to audit failures before choosing what to build.

Adding harness engineering to an existing AI workflow

I would define completion before adding autonomy

I would start with one narrow workflow and write down what successful completion means. For an article-summary assistant, my proposed contract would include the requested article identity, the permitted source material, the expected output structure, and the point where a human takes over.

I would also define incomplete outcomes. An unavailable article, failed validation, and exhausted budget should not all become an apparently successful response containing apologetic prose. I would want the application to distinguish those states without asking another model to interpret them.

I would collect failures before changing the prompt

The source recommends testing representative tasks and categorizing failures. I would include normal requests alongside missing records, incomplete tool responses, unavailable dependencies, and requests outside the workflow’s scope. These would be deliberate test cases, not claims about incidents in my own systems.

For each failure, I would identify the earliest point where the application could have detected it. Was necessary context absent? Was the wrong tool selected? Did a tool return unusable data? Did verification accept something it should reject? That classification would decide my next change.

I would validate tool results at the boundary

Here is the kind of small TypeScript boundary I would add first. This is my illustrative implementation, not an API from the source. The injected reader represents an existing application function. I check both the response shape and whether the returned record matches the requested article.

type Draft = { id: string; body: string };

async function readVerifiedDraft(
  requestedId: string,
  readDraft: (id: string) => Promise<unknown>
): Promise<Draft> {
  if (!requestedId.trim()) {
    throw new Error("Article ID is required");
  }

  const value = await readDraft(requestedId);
  if (typeof value !== "object" || value === null) {
    throw new Error("Invalid article response");
  }

  const record = value as Record<string, unknown>;
  if (
    typeof record.id !== "string" ||
    record.id !== requestedId ||
    typeof record.body !== "string" ||
    !record.body.trim()
  ) {
    throw new Error("Article verification failed");
  }

  return { id: record.id, body: record.body };
}

I would let the orchestration layer handle these failures explicitly. I would not catch an exception here and substitute an empty article. Equally, I would not call this complete verification: it checks identity and structure, not factual accuracy, permissions, or whether a generated summary faithfully represents the article.

I would make retry policy narrower than failure policy

The source describes retries, rerouting, and escalation as possible responses to failed verification. I would choose between them deliberately. I would not repeat an unchanged request merely because the previous result failed a structural check. First I would ask whether another attempt has a plausible path to a different result.

For operations that modify content, I would separately review whether repeating the action is acceptable. My initial version would favor a controlled stop over an automatic retry where the outcome of the previous action is uncertain. I would also verify every retry result, rather than treating the retry itself as recovery.

I would enforce budgets before spending

The source’s prose recommends checking the budget before each model call. Its simplified pseudocode updates the total after an action. I would follow the pre-call rule in production: account for spending already incurred and check the next call’s allowed usage before starting it.

I would keep financial limits separate from step and retry limits. My stop response would record the reason and preserve validated progress for review. I would also test that stopping really prevents another model call; a budget field in a trace would not be enough for me.

I would build traces and evaluations together

I would record the selected tool, validation outcome, retry decision, and usage information for each step. For editorial material, I would prefer identifiers and carefully selected diagnostic fields over indiscriminately copying article bodies into logs. That is my proposed logging policy, not something the source specifies.

Then I would turn observed failures into repeatable evaluation cases. Following the source, I would track task completion, output quality, and cost per task. I would inspect those together: I would not accept lower spending as progress if the workflow simply stopped completing useful work.

Would I add a supervisor agent?

Not initially. The source describes a single agent with verification, a two-agent supervisor arrangement, and multiple specialized agents sharing a harness. I would start with the first pattern unless my evaluations showed a specific need for something more complex.

I would consider a supervisor for errors that structural checks cannot catch and where the consequence of accepting bad output justifies another model call. The source explicitly identifies additional latency and token consumption as trade-offs. I would measure whether the reviewer improves accepted output enough to justify those costs.

I would not treat a second model’s approval as independent proof of correctness. My requirement would remain an evaluation showing which mistakes the review catches and which it still accepts. For publishing decisions, I would retain human approval until that evidence supports a different boundary.

What harness engineering would not solve for me

I would not expect orchestration to repair missing source information or turn a structurally valid answer into a true one. The source separates schema-based verification from semantic checks; that is an important limit. My draft reader above can accept the correct article record while a later summary still misrepresents it.

I would also resist importing the source’s results as forecasts. It reports LangChain improving task completion from 52.8% to 66.5% without changing the model. That supports investigating infrastructure changes, but I would not budget my own improvement around that result without equivalent tasks and evaluation conditions.

If I were building a narrow, read-only assistant with no tools or autonomous continuation, I would skip the elaborate architecture. I would still define acceptable outputs and test them, but I would not introduce supervisors and shared agent services just to use the terminology.

My next step would be smaller: pick one existing workflow, identify the first unchecked boundary, and add a failing test plus an explicit stop condition there. I would want evidence that the application rejects bad intermediate results before giving the agent more things to do.