Context Lifecycle: Compaction, Caching, and the Event Log

Reading time ~16 min · Last reviewed Sep 2026 · Quiz at the end

Lesson 5 treated context as a snapshot: on this call, what should the model see? That question is necessary but it describes a single frame of a film. A real agent makes dozens or hundreds of calls in one trajectory. Every tool result gets appended, every assistant turn gets appended, and the window that was a tidy 12,000 tokens at step one is 140,000 tokens at step sixty. Somewhere along the way three things have to be decided, and most harnesses decide them by accident: where the true record of the session lives, how the growing context is serialized so the provider can reuse work from the previous call, and what happens when the window finally runs out.

This lesson covers those three decisions as one subject, because they are one subject. The event log determines what you can safely throw away. The cache determines what you can afford to change. Compaction is the moment you are forced to change it. Get the relationship right and long trajectories become cheap, resumable, and auditable. Get it wrong and you get an agent that is slow, expensive, and quietly forgets the one constraint the user stated at step three.

Pillar 1: the append-only event log

Model-visible means logged

The simplest harness keeps a messages array in memory, appends to it, and sends it to the model. When the window fills, it rewrites the array. The array is simultaneously the working state, the history, and the audit trail, and every edit to it destroys information. That design works for demos and fails in three predictable ways: you cannot resume after a crash, you cannot explain after the fact why the model did what it did, and you cannot undo a bad compaction.

The alternative is to separate the record from the view. DeepSeek Harness's architecture docs (2026) state the principle as a rule: "Model-visible means logged." Every input that can reach the model must first exist as an event in an append-only session log, and a runtime invariant checks that each model request can be reconstructed from that log. The history the model sees is derived from the log by a projection function, and plugins that alter existing message content must register pure projections rather than mutating what was recorded. Fork, resume, transcripts, telemetry, and persistence are all derived from the same durable events. Even failed, retried, and cancelled model attempts are kept, recorded as log-only attempt events that never enter the model's history. That last detail matters more than it looks: when an eval fails, you want to see the three attempts the harness threw away, not just the one it kept.

Anthropic arrived at the same shape from the infrastructure side. In Scaling Managed Agents (2026), the session is a durable log that lives outside Claude's context window, and the harness reads it through an interface that returns positional slices of the event stream, so it can pick up where it left off, rewind, or reread a specific moment. The harness itself is stateless: if one crashes, a new one boots from the session ID and recovers from the log. The post also names the underlying problem directly, noting that irreversible decisions to keep or discard context can cause failures, because it is hard to know which tokens a future turn will need.

What the log buys you

The replay row connects directly to lesson 17 and lesson 21: an improvement loop that cannot replay real trajectories through a candidate harness is working from anecdotes. The audit row is the one leaders care about. A regulator asking why an agent approved a claim wants the context at the moment of decision, not a reconstruction.

Pillar 2: caching is an architectural constraint

Why the prefix is sacred

An agent loop resends the whole conversation on every step. Without reuse, that makes the total tokens processed over a trajectory grow quadratically with its length. Providers solve this with prompt caching, also called KV caching or prefix caching: if the start of your request exactly matches the start of a recent request, the provider reuses the computed attention state for that shared prefix and only processes the new suffix. OpenAI's Unrolling the Codex agent loop (2026) puts it plainly: with cache hits, "sampling the model is linear rather than quadratic." The same post explains that Codex deliberately constructs each new prompt so that the old prompt is an exact prefix of it.

The operative word is exact. Anthropic's prompt caching docs define the prefix order as tools, then system, then messages, and state that changes at each level invalidate that level and every level after it. Change one byte in the tool definitions and nothing downstream is reusable. The cache also expires: the default lifetime there is five minutes, refreshed on each use, with a paid one-hour option. An agent that pauses for ten minutes waiting on a human approval may come back to a cold cache.

Manus made this the organizing principle of its harness. In Context Engineering for AI Agents (2025), the team calls KV-cache hit rate "the single most important metric for a production-stage AI agent." Their reasoning is the shape of agent traffic: they report an average input-to-output ratio of around 100:1, so almost all of the cost and prefill latency sits in input tokens, which is exactly what the cache discounts.

The cache-safe rules

Stable head. Put what never changes first: system doctrine, tool definitions, stable task context. Put what changes last. Manus names the classic mistake: a timestamp, especially one precise to the second, at the top of the system prompt. It makes every request's prefix unique and silently zeroes your hit rate. If the model needs the time, append it as a message near the end of the context.

Append-only history. Never edit an earlier message to reflect new state. Codex handles mid-conversation configuration changes, such as a new sandbox mode, approval mode, or working directory, by appending a new message in the same format as the original rather than modifying the earlier one. DeepSeek Harness applies the same rule to its own system prompt, which it treats as a logged "surface node": on routes that support it, prompt and tool updates are appended after the cached history instead of rewriting the head, and a new cache lineage (a "request series") starts only by explicit decision (architecture docs).

Deterministic serialization. Identical logical content must produce identical bytes. Manus warns that many languages do not guarantee stable key ordering when serializing JSON, which can break the cache silently. The same applies to tool lists. Codex's post describes an early MCP bug where tools were not enumerated in a consistent order, causing cache misses. The protocol has since absorbed the lesson: the MCP 2026-07-28 changelog says servers SHOULD return tools/list results in a deterministic order "to enable client-side caching and improve LLM prompt cache hit rates."

Mask, don't mutate, the action space. The tempting way to restrict an agent in a given state is to remove tools from the request. That changes the head and invalidates everything after it. Manus instead keeps tool definitions fixed and uses a context-aware state machine with response prefill to constrain which actions the model can choose. The Codex post adds the MCP wrinkle: a server can announce mid-session that its tool list changed, and honoring that notification in the middle of a long conversation can cause an expensive cache miss. A harness should decide when to absorb such a change, for example at the next compaction boundary, rather than letting a remote server decide for it. Tool design itself is lesson 11's subject; the point here is that the tool list is part of the prefix.

Keep harness metadata out of the model-visible prefix. Session IDs, trace headers, plugin inventories, and telemetry do not belong inside the messages. DeepSeek Harness's wire extensions are HTTP headers and additive request fields that remain outside messages, system prompts, and tool schemas, so they add no model-input tokens and do not alter the model-visible prefix. The general rule: if the model does not need to read it, it should not be in the part of the request the model reads.

A worked cost illustration

The numbers below use a hypothetical base input price of $2.00 per million tokens. The multipliers are the ones Anthropic documents for its standard models in the caching docs linked above (cache writes at 1.25x base for the five-minute tier, cache reads at 0.1x base); other providers price differently, so treat the result as a shape, not a quote. Output tokens are ignored and the cache is assumed never to expire.

The trajectory: 50 steps (Manus reports a typical task takes around 50 tool calls), a 10,000-token stable head of system prompt plus tools, and 2,000 new tokens appended per step. Step k sends 10,000 + 2,000 × (k − 1) tokens, so the whole run sends about 2.95 million input tokens.

Two things jump out. The cache-friendly harness is roughly seven times cheaper on input than the uncached one, for the same model and the same work, and the gap widens as trajectories get longer, since the uncached cost grows quadratically and the cached cost nearly linearly. And the broken configuration is worse than not caching at all, because you pay the write premium on every request and get nothing back. That is why a regression in hit rate deserves an alert, not a quarterly review. Latency follows the same curve: cached prefill is fast, so time-to-first-token on step 50 stays close to step 5. The full economics of agents, including output tokens, retries, and model routing, are lesson 26's territory.

A cautionary tale

Cache behavior and context management interact in ways that escape normal testing. Anthropic's April 23 postmortem (2026) describes a Claude Code change meant to save cost and latency when resuming idle sessions by clearing old thinking blocks once. A bug made the clearing happen on every turn for the rest of the session. Users saw forgetfulness, repetition, and odd tool choices as the model lost its own earlier reasoning, and every turn became a cache miss, so usage limits drained faster. The change shipped on March 26 and was fixed on April 10. The postmortem notes the bug sat at the intersection of context management, the API, and extended thinking, and slipped past code review, unit tests, end-to-end tests, and dogfooding.

The lesson for harness engineers is not "be more careful." It is that context-lifecycle changes need their own instrumentation. Track cache hit rate per step as a first-class metric, alongside the quality evals from lesson 18. A sudden drop in hit rate would have been a loud, cheap signal for a bug whose quality symptoms were subtle and easy to blame on the model.

Pillar 3: compaction

Eventually the window fills. Compaction is the harness replacing part of the model-visible history with something smaller so the trajectory can continue. It is the one place where the harness deliberately breaks the append-only, stable-prefix discipline of Pillar 2, which is why it should happen rarely, in large deliberate steps, and with the original preserved in the log from Pillar 1.

Triggers

There are three, and a production harness wants all of them. Token pressure: before each step, measure the projected context and compact when it crosses a threshold, leaving headroom for the next tool result and the model's output. Overflow error: the provider rejects a request as too long, so compact and retry. Manual: a user or operator runs a /compact command, typically at a natural boundary between subtasks. DeepSeek Harness implements exactly this trio. Its agent lifecycle docs (2026) describe pressure checks before request derivation, a separate path that fires only on a canonical context-overflow error, and a retry that happens only when compaction actually advanced the context; otherwise the original error stands. That last guard prevents an infinite compact-retry loop. Codex followed a similar path: its first compaction was a manual /compact command, and it now compacts automatically once a configured limit is exceeded, using a dedicated Responses API endpoint (Codex, 2026).

Ordering: cheap and restorable before lossy

Not all context bloat is equal. In most coding and research agents the biggest items are tool results: a 40,000-token log file, a full web page, a directory listing. Those can usually be shrunk without a model call and without losing anything permanently, because the full content can live somewhere retrievable. DeepSeek Harness orders its compaction accordingly: once a trigger fires, optional tool-result pruning runs before any summary is attempted, the context is remeasured, and pruning alone can resolve the pressure (compaction subsystem reference). A separate spill package stores oversized tool output outside the model's context and returns a locator with retrieval guidance in its place.

Manus describes the same idea as restorable compression: drop the body of a web page as long as the URL is preserved, drop a document's contents as long as its file path remains, so the agent can fetch it again if it turns out to matter. Only after the restorable options are exhausted should the harness pay for a lossy, model-written summary.

What must survive

A summary is a lossy compression performed by a model that does not know what the next hundred steps will need. So tell it. At minimum, a compaction summary must carry forward:

• The goal and constraints, in the user's words where possible. Paraphrase drift on the original request is the most expensive failure.
• Decisions made and why, including approaches tried and rejected. Without them the agent re-explores dead ends.
• Open todos and what "done" means for each.
• Concrete references: file paths touched, identifiers, URLs, commands that worked. These are cheap to keep and impossible to reconstruct from prose.
• Errors seen. Manus argues for leaving wrong turns in context so the model does not repeat them; a summary that erases failures erases that benefit.

Two structural rules sit alongside the content rules. Keep a recent tail verbatim, since the last few steps are what the model is actively reasoning about. And never split a tool call from its result; DeepSeek Harness's compaction preserves tool-call and result pairing at range boundaries. In that harness the summary itself is logged along with the exact range it replaced, and the original events stay in the log, shadowed rather than deleted. That is Pillar 1 paying off: a bad summary is a recoverable incident, not permanent amnesia.

A compaction policy

The pseudocode below puts the pieces together. It is Python-shaped and deliberately small; the structure is what matters.

def prepare_step(session, budget):
    view = project(session.log)                # the window is a view over the log
    if tokens(view) < budget.soft_limit:
        return view

    # 1. Restorable first: spill oversized, older tool results
    for ev in view.tool_results(older_than=budget.keep_tail_steps):
        if tokens(ev) > budget.max_inline:
            ref = spill_store.save(ev.content)     # full text kept outside the window
            session.log.append(Replace(ev.seq, preview(ev, ref)))
    view = project(session.log)
    if tokens(view) < budget.soft_limit:
        return view

    # 2. Lossy second: summarize a closed older span, keep the tail verbatim
    span = select_span(view, keep_tail=budget.keep_tail_steps,
                       keep_tool_pairs=True)
    summary = summarize(span, must_keep=["goal", "constraints", "decisions",
                                          "open_todos", "file_paths", "errors"])
    if not covers(summary, span.file_paths + span.todo_ids):
        raise CompactionRejected(span)         # fail loudly, don't lose facts
    session.log.append(Summary(span.range, summary))   # originals stay in the log
    session.start_request_series("compaction") # one deliberate cache break
    return project(session.log)

def on_request_error(err, session, budget):
    if err.kind != "context_overflow":
        raise err
    gen = session.surface_generation
    prepare_step(session, budget.emergency)
    if session.surface_generation == gen:      # no progress: don't loop
        raise err
    return RETRY

Note the covers check. A summary validated against the mechanical facts of its span, such as every file path and todo ID that appeared, catches the most common silent loss cheaply. It will not catch a dropped nuance, which is why compaction quality also belongs in your eval suite.

Recitation: keeping the goal in recent attention

Compaction handles volume. A related problem is attention: in a long trajectory the original goal is far behind the model, buried under tool output. Manus's answer is recitation. The agent maintains a todo.md file and keeps rewriting it as it works, so the overall plan is restated at the end of the context, where the model's attention is freshest. Anthropic's Effective harnesses for long-running agents (2025) uses durable files for the same purpose across whole context windows: a progress file that logs what agents have done, a feature list, and git history, which each fresh session reads to get up to speed.

Recitation and compaction complement each other. A todo file that lives on disk survives any compaction by construction, and a summary that references it can stay short. This is the first appearance of an idea lesson 8 develops fully: some state belongs outside the window altogether, in a tier the agent reads on demand.

Resets versus compaction

A context reset is the blunt alternative: end the session, start a fresh one, and hand over only structured artifacts like the progress file. Compaction preserves continuity; a reset gives a clean slate. Anthropic's Harness design for long-running application development (2026) documents both sides. With Claude Sonnet 4.5, which showed "context anxiety" (wrapping up work prematurely as it believed it was nearing its limit), compaction alone was not enough and resets became essential. After moving to Opus 4.6, the author was able to drop context resets entirely and run one continuous session across the whole build, with the Claude Agent SDK's automatic compaction handling context growth.

That is a useful data point, not a law. The same post's broader argument is that harness components encode assumptions about what the model cannot do, and those assumptions expire. Resets were a workaround for a model behavior, and when the behavior faded, the workaround became overhead. Keep your lifecycle policy configurable per model, and re-test it at every model upgrade. Lesson 15 goes deeper on multi-session work.

The connection to lesson 5's failure modes

Lesson 5 named "stale summaries," where details drop out after several summary-of-a-summary cycles, and "context collapse," where repeated full rewrites erode accumulated detail. Compaction is the machine that produces both. Each compaction that summarizes a previous summary is another lossy generation, and the loss compounds. Three defenses follow from this lesson. First, summarize from the log, not from the previous summary, when you can afford it: the originals are still there. Second, move durable facts into files the agent maintains (recitation) so they never pass through the summarizer. Third, validate summaries mechanically and evaluate them on real trajectories, the same way you would evaluate any other model output.

Putting the pillars together

The three pillars form one design. The event log records everything, so the window can safely be a lossy view. The cache rewards a view that only grows at the end, so the harness appends and keeps its head stable. Compaction is the rare, deliberate moment the view is rebuilt: restorable reductions first, a validated summary second, a single intentional cache break, and the originals still in the log. DeepSeek Harness's internals, which implement all three pillars in one codebase, are worth studying in full; lesson 23 does that teardown. How the loop that drives all this steps from call to call is lesson 4.