Anatomy of a Harness

Reading time ~12 min · Quiz at the end

Lesson 1 defined the harness as everything wrapping the model. That definition is correct but too coarse to build from. If you tell an engineering team "go build a good harness," you'll get seven different interpretations and a system with three subsystems built well and four missing entirely — usually the four nobody thought to name. This lesson names them. A harness decomposes into seven subsystems, each answering a distinct question, each failing in a distinct and recognizable way when it's absent or weak.

The value of naming them separately is diagnostic. When an agent misbehaves in production, "the AI is broken" is not an actionable statement. "The context assembly step is retrieving the wrong policy version" is. Every incident you'll ever debug in an agent system traces back to exactly one or two of these seven subsystems, and knowing the map cuts your diagnosis time from days to minutes.

The seven subsystems

Worked example: an insurance claims agent

Consider a first-notice-of-loss agent for an auto insurer. A customer reports a fender-bender through a chat interface. The agent needs to gather facts, check the policy, estimate whether the claim is straightforward or needs adjuster review, and either fast-track a payment or route to a human. Walking this through all seven subsystems makes each one concrete. Orchestration loop. The agent doesn't answer in one shot — it runs a loop: ask a clarifying question if the accident description is incomplete, call the policy-lookup tool, call the coverage-check tool, decide whether it has enough information to recommend fast-track or escalation, and either respond to the customer or hand off to an adjuster queue. Without a well-designed loop, the agent might try to fast-track a claim before it has confirmed the policy is even active, or ask the customer the same question three times because it isn't tracking what it has already learned within the conversation. Context assembly. On each turn, the agent needs the current policy terms (not last year's, not a generic template), the specific coverage type relevant to this loss, and the conversation so far — and nothing else. If context assembly is naive and dumps the customer's entire policy history plus five unrelated product brochures into every call, the model's attention gets diluted and it becomes more likely to cite a coverage limit from the wrong policy year. Memory. If the customer starts the claim on the app, gets interrupted, and resumes by phone an hour later, the case state — photos already uploaded, questions already answered — needs to persist across that channel switch. Without memory, the customer re-answers everything, and re-answering after a car accident is exactly the kind of friction that turns a mildly annoyed customer into a complaint on a review site. Tools and action surfaces. The agent needs a policy-lookup API, a coverage-rules engine, possibly a photo-damage-estimation service, and a claims-system write action to actually file the claim. Each of these needs a clear contract: what inputs it expects, what it returns on success, and critically, what it returns on failure (policy not found, system timeout, ambiguous match). An agent whose tool contract doesn't specify failure behavior will often hallucinate a plausible-sounding result rather than surface the failure. Guardrails and permissions. This is where policy meets code. The agent might be authorized to fast-track payments under $2,000 with no injuries reported and a clean policy history, and required to escalate everything else to a human adjuster. That threshold is a business decision encoded as a hard constraint, not a suggestion in a prompt. Without an enforced guardrail, "the agent should escalate claims over $2,000" is a sentence the model might follow 97% of the time — and the 3% miss rate is exactly the kind of thing that shows up in a regulatory exam. Evaluation and feedback loops. Before any change ships — a new prompt, an upgraded model, a new retrieval source — it runs against a held-out set of historical claims with known correct outcomes. If a change causes the fast-track/escalate decision to flip on claims it used to get right, that shows up in the eval run, not in next month's loss ratio. Economics and telemetry. Each claim conversation costs some number of tokens across however many model calls the orchestration loop makes, plus whatever the photo-estimation service charges per call. At 200 claims a day this is a rounding error. At the volume of a national carrier's storm-surge day — tens of thousands of claims in 48 hours — the same per-claim cost and the same per-claim latency need to have been modeled in advance, or the system either falls over or the invoice becomes a board-level conversation.

A second pass: the same seven subsystems in field operations

The insurance example makes the seven subsystems concrete, but it's worth seeing them once more in a different domain, because the specific shape each subsystem takes changes even though the underlying question doesn't. Consider a utility company's crew-dispatch agent, which recommends which field crew to send to a reported outage or service request, factoring in crew certifications, current location, equipment on the truck, and job priority. Orchestration here looks different from the claims example — there's less back-and-forth conversation and more of a pipeline: ingest the incoming work order, pull crew availability, score candidate crews against the job's requirements, and either auto-assign or flag for a dispatcher to decide. The loop's key decision point isn't "do I have enough information from the customer," it's "is my confidence in this crew match high enough to auto-assign, or does this need a human dispatcher's judgment." Context assembly means the agent needs the work order's technical requirements, the certifications and equipment manifest for candidate crews, and current traffic or access conditions — not the crew's entire employment history or every job they've ever completed. Memory matters across a shift: if a crew already had two emergency calls diverted to them this morning, that fact needs to persist into the next dispatch decision so the same crew doesn't get triple-booked. Tools and action surfaces mean a real-time crew-location API, a certifications database, and a dispatch-system write action, each needing a defined contract for what happens when, say, the location service is down for a crew mid-route. Guardrails encode real safety constraints, not just business policy: a crew without a confined-space certification must never be auto-assigned to a confined-space job, full stop, enforced in code, because the failure mode here is a safety incident, not a customer complaint. Evaluation means testing dispatch recommendations against a set of historical jobs where the actual outcome (on-time, needed a second crew, safety near-miss) is known, so a change to the scoring logic gets checked against real dispatch history before it goes live. Economics and telemetry means knowing the cost and latency of a dispatch decision at one request per outage versus the volume spike during a regional storm event, when hundreds of outages hit within an hour and the dispatch agent needs to keep pace without degrading into slow, expensive, sequential calls. The domains are unrelated — insurance claims and utility crews share no data model, no regulatory regime, and no customer interaction pattern — yet the seven-subsystem decomposition applies unchanged. That consistency is what makes the framework useful as a diagnostic tool across an entire portfolio of agent projects, not just a one-off checklist for a single use case.

The license-versus-deployment distinction

A pattern shows up repeatedly in enterprise AI purchasing: an organization licenses an agent platform — a vendor's claims-automation suite, a customer-service agent product, a CRM copilot — and treats the purchase order as the finish line. Six months later, the system is live but underused, adjusters don't trust its recommendations, and the steering committee is asking why the ROI case hasn't materialized. What happened is usually that the platform shipped with generic versions of these seven subsystems, or with some of them left as configuration stubs for the customer to fill in — and no one filled them in. The platform's default context assembly doesn't know your policy taxonomy. Its default guardrails don't encode your specific delegation of authority. Its default eval suite, if it has one at all, was built on the vendor's synthetic data, not your historical claims.

Subsystems interact — they are not independent

It's worth being explicit that these seven are a decomposition for clarity, not seven unrelated boxes. A guardrail is often enforced by intercepting a tool call (subsystems 4 and 5 overlapping). An eval suite needs realistic context assembly to test against (6 and 2). Memory design affects what context assembly needs to fetch fresh versus what it can pull from state (3 and 2). When you're diagnosing a production incident, expect to find the root cause at the boundary between two subsystems as often as within one — a classic example is an agent that "ignores" a guardrail instruction because context assembly buried it beneath eight thousand tokens of retrieved documents, which is really a context problem wearing a guardrails costume.

Incident: Discount agent approved 22% off, policy caps at 15%
Trace:
  1. Guardrail existed: system prompt states "never exceed 15% without approval"
  2. Context assembly: prompt instruction was 40 lines above 3,000 tokens
     of retrieved deal history
  3. Root cause: not a missing guardrail — a context assembly failure
     that buried the guardrail below the model's effective attention span
  4. Fix: move hard constraints to a structured, enforced check outside
     the model's discretion (deterministic code), not prompt text alone

The harness as an operating system

A useful way to hold all seven subsystems in your head at once is to borrow an analogy from systems software. A harness, like an operating system, earns its keep by encapsulating complicated logic behind a simple interface — the model, like an application, should not have to know how memory is paged, how a tool call is scheduled, or how a permission is checked, only that a clean interface exists to ask for what it needs. Weng (2026) makes this comparison explicit and draws a design lesson from it: harness design should be deliberately simple and generic so that it generalizes across tasks rather than overfitting to one, and it should lean on existing software-engineering conventions wherever it can. The payoff for leaning on conventions is specific and underrated — a model has seen an enormous amount of ordinary software during pretraining, so a harness built out of familiar shapes (a filesystem, a shell, a git repository, a JSON config) lets the model bring that pretraining knowledge to bear instead of forcing it to learn your bespoke abstractions from a few in-context examples.

This is also why the surface keeps converging. The configs, tool interfaces, and protocols that a harness exposes are gradually standardizing across the industry — not because a committee mandated it, but because the generic, convention-following designs are the ones that generalize, and the idiosyncratic ones quietly lose. When you find yourself inventing a novel abstraction for something the rest of the field already does with a file and a shell command, that is usually a signal to stop inventing.

The most important of those conventions is the filesystem, and it deserves to be called out because it cuts directly against a mistake almost every team makes early: trying to carry the whole workflow and all of its logs inside the context window. That does not scale, and it does not need to. The durable state of a long-running agent — experiment logs, code diffs, error traces, the trajectories of past runs — routinely grows far beyond even a generous trained context length, so it belongs in files, not in the prompt. This is the filesystem-as-persistent-memory pattern, a recurring design choice in long-horizon agent systems, and it connects the abstract "Memory" subsystem above to something concrete you already know how to build. It also has a compounding advantage: reading, writing, and editing files (usually through bash) is a foundational skill that models are trained heavily on, so a file-based memory design automatically gets better as core model capability improves, with no change to your harness. Lesson 15 leans hard on exactly this property when it treats accumulated trajectories on disk as the substrate a system can search over and improve against.

The convention that has stabilized fastest is the coding-agent tool surface. Across Claude Code, Codex, OpenCode, and Cursor-style agents, the set of actions a capable agent is given has converged to a recognizably common shape, summarized below. Treat the table as illustrative rather than exhaustive — the exact names and groupings differ per system — but the categories are stable enough that if your harness is missing an entire row, that absence is worth a deliberate justification rather than an oversight.

Seeing the surface laid out this way also clarifies what harness engineering is and isn't. The early framing of an agent as "an LLM plus memory plus tools plus planning" captures the top two rows of that table and little else. Harness engineering, as this course uses the term and as Weng (2026) frames it, additionally owns the workflow and loop design, the evaluation, the permission controls, and the persistent-state management — which is why it sits much closer to runtime and software-system design than to prompt templating. The seven subsystems are simply that same claim, decomposed.

Where each subsystem is covered later

This course spends an entire lesson or more on nearly every subsystem named here, because each one is a discipline in its own right:

• Context assembly — lesson 4 (context engineering) and lesson 5 (context graphs and organizational judgment).
• Memory — lesson 6 (memory tiers: session, agent, and organizational).
• Tools / action surfaces — lesson 9 (tools, MCP, and action surfaces), building on lesson 7's treatment of specs.
• Guardrails & permissions — lesson 10 (guardrails and governance).
• Evaluation & feedback loops — lesson 11 (observability versus evals), lesson 12 (building eval harnesses), and lesson 13 (review architecture).
• Economics & telemetry — lesson 17 (agent economics: pricing, tokens, and pods).
• Orchestration loop — touched throughout, with dedicated treatment in lesson 18 (multi-agent orchestration patterns) for anything beyond a single loop.

Lesson 3, immediately next, zooms into the one component that sits at the center of all seven subsystems without being synonymous with any of them: the model itself. Understanding what a model is actually good at, and how that capability arrives, is what lets you make sane tier and vendor decisions instead of chasing leaderboard rankings.