Self-Improving Harnesses: Search, Evolution, and RSI

Reading time ~18 min · Last reviewed Sep 2026 · Quiz at the end

Lesson 21 ended with a human — or a closely supervised model — turning the crank on the improvement loop: propose a change, run the frozen eval, commit or revert, log. This lesson asks the question that loop was always building toward: what happens when the harness itself becomes the object of automated search? Not a model suggesting prompt tweaks for a human to review, but a system in which the code that orchestrates prompts, tools, memory, and control flow is proposed, evaluated, and merged by the machine — sometimes thousands of candidates deep. This is no longer speculative. There is now a real research literature, with real results and real cautionary tales, and a practicing harness engineer who hasn't read it is designing by hand in a domain that is becoming a search problem.

The intellectual lineage is older than the field. I. J. Good (1965) imagined an "ultraintelligent machine" that could design better machines than itself; Yudkowsky (2008) named the feedback loop recursive self-improvement (RSI): an AI using its current intelligence to improve the machinery that produces its intelligence. What the recent literature makes concrete — and what Weng (2026) argues persuasively — is that the near-term version of this loop does not start with a model rewriting its own weights. It starts with the model improving its deployment system: the harness. The layer this whole course is about turns out to be the first rung of the RSI ladder, which is either exciting or sobering depending on how good your evals are. By the end of this lesson you should understand both readings.

The optimization ladder

There is a clean progression in what gets optimized as models get stronger, and it organizes everything else in this lesson: instruction prompts → structured context → workflows → harness code → optimizer code. Each rung subsumes the one below it. Early prompt engineering optimized strings. Context engineering (lesson 5) optimized what the model sees per call. Workflow design optimizes the graph of calls. Harness-code optimization treats the entire orchestration program — prompts, tool wiring, memory logic, control flow, subagent policy — as one editable artifact. And at the top, optimizer-code optimization improves the machinery that improves the harness: the improver improving the improver.

Notice what this ladder implies about the skills in this course. Lessons 5 through 20 taught you to design each layer by hand. This lesson's literature says: hand design is how you build the first version and the evaluation; search is increasingly how the later versions get found. The engineer's job migrates up the stack — from writing the harness to writing the search space, the fitness function, and the guardrails around the loop. That is a bigger shift in job description than it first appears, and it is the same shift lesson 21 described in miniature.

Searching the workflow layer

Workflows were the first harness layer to be formally searched, because a workflow is easy to represent and cheap to evaluate on benchmark tasks. ADAS (Hu et al., 2025) — Automated Design of Agentic Systems — made the move explicit: maintain an archive of agentic workflows seeded with simple patterns (chain-of-thought, self-refine), then ask a meta-agent to program new agent designs in code, inspired by what's already in the archive. Each candidate is drafted as a high-level description, implemented as a program, passed through self-refinement checks for novelty, evaluated, and added back to the archive if it earns its place. The archive is the population; the meta-agent is the mutation operator; the eval is selection.

AFlow (Zhang et al., 2025) tightened the search. It represents a workflow as a graph — nodes are LLM-invoking actions, edges are logical operations implemented in code — and drives the search with Monte Carlo Tree Search: select a promising workflow node with a soft mix of score and exploration, ask an LLM to produce a modification conditioned on evaluation feedback, execute, evaluate, keep it if it improves within budget, stop when the top candidates plateau. On QA, code, and math tasks, AFlow beat both manually designed workflows and ADAS. Read that sentence again from the perspective of someone whose job title is "designs workflows manually": on well-evaluated tasks, the search already wins.

Domain experts still matter — as designers of the initial structure and the constraints. The auto-research systems that work, like the AI Scientist (Lu et al., 2026) pipeline (propose ideas, write code, run experiments, analyze, write up, peer-review) or ScientistOne (Meng et al., 2026) — which makes verifiability the central design constraint, requiring every citation, number, method, and conclusion to trace to an evidence source audited by chain-of-evidence checks — are expert-designed workflows at the top level, with search and iteration operating inside them. The lesson for practitioners is not "stop designing workflows." It is: design the workflow as a search space with explicit degrees of freedom, not as a finished artifact.

Evolving the harness itself

The workflow is one component. The stronger move is to put the whole harness codebase under evolutionary search. Evolutionary methods fit this problem for two reasons: the search space is enormous and weirdly shaped (no gradients through "reorganize the memory subsystem"), and candidate solutions are easy to evaluate if — and only if — you have a fast, objective fitness signal. The lineage runs from prompt evolution (Promptbreeder, where even the mutation instructions evolve; GEPA, which uses natural-language reflection over trial-and-error trajectories to propose prompt updates) up through program evolution to full harness evolution.

AlphaEvolve (Novikov et al., 2025) is the canonical program-evolution system: a pool of candidate programs, frozen LLMs prompted with parent programs and their results to generate diffs, repeated evaluation, survival of the fitter. Three design details deserve a harness engineer's attention. First, the editable surface is explicitly bounded — regions open to modification are marked with EVOLVE-BLOCK comments, and the agent sees the full repo but may only change what's marked. Second, the prompt fed to the mutation LLM is itself co-evolved alongside the solutions. Third, ablations show each piece — the evolutionary procedure, the context in prompts, the meta-prompts, and stronger base LLMs — contributes measurably. ShinkaEvolve (Lange et al., 2025) adds sample-efficiency machinery: parent sampling balanced between performance rank and offspring count, embedding-based rejection of candidates too similar to the existing population, and a meta-scratchpad that accumulates patterns from successful solutions to guide future mutation. That last item should sound familiar — it is the experiment log from lesson 21, promoted from bookkeeping to an active steering mechanism.

The Darwin Gödel Machine (DGM; Zhang et al., 2025) closes the loop this lesson is named for: a coding agent that is allowed to modify its own harness repository. Start with one agent in the pool; each iteration, pick a parent (proportional to performance, inversely to how many children it already has), let it read its own benchmark evaluation logs, propose improvements to its own harness codebase using nothing fancier than bash and a file editor, and admit the offspring to the pool only if it scores well enough. With Claude 3.5 Sonnet as the fixed base model and a deliberately simple starting harness, DGM-discovered agents went from 20% to 50% on SWE-bench Verified and 14.2% to 30.7% on Polyglot — reaching or beating handcrafted agents. Hold that result against this course's central slogan: the model was a constant; every point of that gain was harness. A follow-up, Hyperagents (Zhang et al., 2026), adds a meta-agent that decides how to modify existing task agents to create new ones — the optimizer-code rung of the ladder.

Context-layer search follows the same trajectory, one level down: lesson 5's new material covers ACE's evolving context playbook and MCE's bi-level split between context-management mechanism and context content. Meta-Harness (Lee et al., 2026) goes a rung further: the optimized object is the code that decides what gets stored, retrieved, and presented to the model — a harness for optimizing harnesses. Its proposer is itself a coding agent; every proposed harness lives in the filesystem as a directory of source code, scores, rollout trajectories, and state updates, and the proposer reads the full execution history with grep and cat rather than shoveling it into one giant prompt. The output is not a single winner but a Pareto frontier of harness candidates. Once harness design becomes an executable search space, a strong coding agent exploits the same design space human engineers use — and it does not get tired on candidate number four hundred.

The capability threshold, and improving the improver

Lesson 21 introduced the cautionary result from STOP (Zelikman et al., 2023), and it bears repeating in this lesson's terms because it is the single most decision-relevant finding in the literature: recursive self-improvement of the improver worked with GPT-4 and degraded with weaker models. The recursive structure is not the magic; the base model's capability is. A self-improvement loop wrapped around a model below the threshold doesn't idle — it actively erodes the harness while producing locally plausible-looking changes at every step. The practical test before you automate any part of your improvement loop is therefore not "is the loop well designed?" but "is the model strong enough to improve this mechanism, as demonstrated on a frozen eval?" — and the honest answer changes with every model generation, which is why the loop's governance cannot be a one-time decision.

Self-Harness (Zhang et al., 2026) shows what production-grade governance of the loop looks like (lesson 21 covers its held-in/held-out, no-regression acceptance rule in detail). Two aspects matter here. First, its edits are bounded: the proposer is told exactly which surfaces of the harness are editable, is fed verifier-grounded failure patterns plus records of passing behavior that must be preserved, and is steered toward recurrent, addressable error patterns fixable by narrow changes. Second, the results were model-specific: running the loop over different base models produced different harness instructions targeting each model's distinct weaknesses. That is a quietly important result for anyone maintaining a harness across model upgrades — the optimal harness is a function of the model underneath it, which is an argument for keeping the adaptation loop alive rather than treating harness design as a finished project.

The frontier beyond harness-only adaptation is joint optimization of harness and weights. SIA (Hebbar et al., 2026) is an early attempt: a meta-agent proposes the initial harness, a task agent executes, and a feedback agent decides — from recent trajectories — whether the next iteration should update the harness or the model weights. Weng's assessment, which is worth adopting wholesale as a habit of reading this literature: the direction is interesting, the evidence is provisional (the task agent was far weaker than the meta and feedback agents, and the baselines were too weak to cross-reference cleanly), and stability and Goodhart problems remain open. Not every paper in a hot area is a result.

2026: harness search goes mainstream

Through 2025 the systems above were mostly research demonstrations on algorithmic tasks. In 2026 harness search turned into a crowded field, pointed straight at the agent benchmarks practitioners actually cite, and the methods diversified in what they search. Read the table as a map of design choices. Each row fixes the model (except the last) and varies a different slice of the harness.

Four patterns in this table matter more than any single number.

Structure beats prose. AHE's single-component ablation is the most useful result in the table for anyone deciding where to spend harness effort. Swapping in only the evolved long-term memory lifted Terminal-Bench 2 from 69.7% to 75.3%; tools alone to 73.0%; middleware alone to 71.9%. The evolved system prompt alone regressed to 67.4%. The paper's reading is that structural changes transfer better than prose-level strategy edits. Recursive Harness Self-Improvement points the same way from a different angle, finding that gains came from context management and information flow between agents rather than from more reasoning. If your team's harness work is mostly rewording the system prompt, the evidence says you're working on the weakest lever.

Edits as falsifiable predictions. AHE's Evolve Agent must justify every edit with failure evidence, a root cause, a targeted fix, and a predicted impact: which tasks should flip to passing and which are at risk. The next iteration's evaluation checks the prediction. This turns lesson 21's "we changed X and the eval moved by Y" into a contract written before the run, which is the automated form of deciding your stopping rule in advance. Its Agent Debugger, which condenses roughly 10M tokens of traces per iteration into layered reports that link back to raw evidence, is the trace discipline from lesson 17 in production form.

Size the search surface to the evaluator. AutoAgent confines edits to one file and lets a human steer through program.md, a deliberately small, reviewable surface. HARBOR shrinks it further to a vector of flags, which makes Bayesian optimization possible and the results easy to audit. JIT-Agent goes the other way and generates a harness per task. The right choice depends on how cheap and trustworthy your evaluator is, the same constraint the evolutionary-search pitfall above describes.

Benchmark gains need disclosure. Nearly every row reports Terminal-Bench or similar scores for a harness that was searched on that benchmark. AHE's transfer to SWE-bench Verified and to other model families is the right kind of check, but read all of these numbers with lesson 19 in hand: which benchmark version, which infrastructure, how many runs, and was the harness frozen before the reported evaluation? A searched harness is overfit to its evaluator until shown otherwise. The same skepticism applies to vendor harnesses shipped alongside a model; lesson 23 tears one down.

What breaks: the failure catalog

The most useful artifact for a practitioner may be the failure taxonomy. Trehan & Chopra (2026) ran LLMs from research idea to paper with minimal scaffolding and basic file tools, and of the ideas selected only one made it to a fully executed paper. Their six recurring failure modes map almost one-to-one onto harness subsystems this course has covered, which is exactly why the catalog is worth memorizing:

To that list, Weng adds loop-level challenges that should read as design requirements for anyone building a self-improving harness. Weak and fuzzy evaluators: self-improvement works where evaluation is measurable and objective, and research taste, novelty, and long-term value are not yet that. Negative results: literature is biased toward successes, so models trained on it are bad at abandoning hypotheses and reporting failure — your harness should make failed attempts easy to preserve, because learning from failure is how the search space gets pruned. Diversity collapse: evolutionary and RL loops exploit known high-reward patterns, and the population degenerates into variants of one solution unless you engineer for diversity — critical in open-ended work where the best path initially scores worse under the current evaluator. Reward hacking: the loop optimizes whatever signal it's given — unit tests get overfit, judge models get gamed, benchmark artifacts get exploited. And long-horizon incentives: a coding agent can complete the task at hand while degrading the long-term health of a shared repo, because sandbox-style training rarely captures maintainability, ownership boundaries, migration cost, or future debugging burden.

Will the harness layer survive?

The honest question every harness engineer should sit with: if models keep improving, does this layer get absorbed into the model? Weng's prediction — and the historical precedent supports it — is a two-part answer. Manual prompt tricks became less central as instruction tuning and model reasoning improved, but the need to specify goals, constraints, context, and evaluation never disappeared. Expect the same here: many current harness mechanisms will be internalized into core model behavior (context management especially — humans maintain memory over a lifetime without a filesystem strapped to their head, and there's no principled reason the capability stays external forever). But the interface to external context, tools, permissions, and evaluation remains, because it is about the world, not the model. The first joint-optimization evidence sharpens this. WHALE (Kim et al., 2026) alternated weight fine-tuning with harness search and found the binding constraint depends on the domain: on SearchQA, harness search matched the best weight-only accuracy with far fewer rollouts, while on mathematical reasoning improvements appeared only after a weight update. Its authors also note the two interact, since weight updates change which harness is effective and harness updates change which capabilities are exposed. So "harness or weights?" is the wrong question. For retrieval-heavy, tool-heavy work the harness is often where the next gain is; for raw reasoning it often isn't; and the best harness for next quarter's model is not this quarter's. Meanwhile the near-term dynamic runs the other way: harness engineering is becoming meta-methodology — improving the machinery for getting better answers, with fewer heuristic rules and more general mechanisms — and mature harnesses are what make model self-improvement loops workable at all, while smarter models keep harnesses from ossifying into overengineering.

The career translation of that paragraph: heuristic patches — the "always retry twice, then reword the prompt" folk wisdom — are depreciating assets that the next model generation makes obsolete or the next search loop rediscovers in an afternoon. Evaluators, telemetry pipelines, permission architecture, editable-surface design, and curated failure archives are appreciating assets: they are exactly the components every method in this lesson consumes as input and none of them can generate for itself. Invest accordingly.

Measuring any of this: the benchmark landscape

Claims about self-improving and research-capable agents need yardsticks, and a small ecosystem of benchmarks has emerged that a harness engineer should know by name — both to read the literature critically and to steal eval-design ideas from (every one of these is an eval harness in the lesson 18 sense, built by people who took grading seriously):

The RE-Bench result deserves a moment: agents dominate at short time budgets and humans overtake at long ones. That is a precise, quantified statement of where the harness frontier is — long-horizon coherence is the open problem, which is why memory, persistent artifacts, and context lifecycle management (Module 2) keep showing up as the binding constraints in every failure analysis in this lesson.

This lesson draws substantially on Lilian Weng's survey "Harness Engineering for Self-Improvement" (Lil'Log, July 2026), which is the recommended companion reading and the source for the research synthesis above.