Specs: The Spec Is the Program

Reading time ~12 min · Quiz at the end

For most of software history, the spec was a courtesy. A product manager wrote a document, engineers skimmed it, and the real specification emerged as code — in pull requests, in Slack clarifications, in the thousand small decisions a developer makes per hour that never made it into any doc. The code was the source of truth because writing code was the expensive, durable act, and English prose was cheap and disposable by comparison.

That relationship has inverted, and it happened quietly enough that a lot of engineering organizations haven't updated their process to reflect it. When an agent can generate a working implementation from a description in minutes, the code is no longer the expensive artifact — it's disposable, regenerable, and increasingly the thing you'd happily throw away and rebuild if the spec turns out to be wrong. The spec is now the expensive artifact, because it's the thing that took the actual thinking, and it's the thing you'll need again the next time you regenerate the implementation, retarget it to a different framework, or hand it to a different agent entirely.

The central asymmetry

Here is the asymmetry that this lesson is organized around, and it is worth stating plainly because it explains almost every spec-related failure this course has seen in production agent work: easy specs produce hard builds, and hardened specs produce easy builds. An easy spec — "build me a tool that flags suspicious insurance claims" — costs its author five minutes to write and imposes hours or days of rework downstream, because every decision the spec didn't make gets made ad hoc, invisibly, by whoever or whatever builds the thing. A hardened spec — one that has actually resolved what "suspicious" means, what evidence counts, what the false-positive tolerance is, what happens at the boundary cases — costs its author real time up front, and the build that follows is close to mechanical. This asymmetry is not new to agents; a experienced engineering lead has always known that a vague ticket costs more in meetings than a precise one costs in writing. What's changed is the leverage. When a human developer hits an unspecified decision point, they usually stop and ask. When an autonomous agent hits the same gap, it doesn't stop — it picks something plausible and keeps going, confidently, and the gap becomes a wrong assumption baked into working code that passes a demo and fails in week three of production.

What a hardened spec actually contains

A hardened spec is not a longer spec. Length and rigor are unrelated; a ten-page requirements document can still fail to resolve a single meaningful edge case, and a page of tight prose with the right worked examples can be airtight. What makes a spec hardened is that it contains five specific things, and a spec missing any one of them will leak decisions downstream to whoever builds it. First, behavioral examples — concrete input/output pairs that show, not tell, what the system should do. "Flag claims with inconsistent damage timelines" is a description; "Given a claim reporting a roof leak dated three days after a storm event with no storm damage reported by neighboring claims in the same ZIP within 72 hours, flag as timeline-inconsistent with confidence tier B" is a behavioral example, and it resolves ambiguity a description cannot. Second, edge cases, stated explicitly rather than left to be discovered. What happens when the claimant has no prior claims history to compare against? What happens when two data sources disagree on the storm date? A spec that only describes the happy path is a spec that hasn't been hardened. Third, acceptance criteria — a test a reasonable third party could run against the output to say pass or fail, independent of how it was built. This is the connective tissue to lesson 12's eval sets: acceptance criteria that are actually executable, not just described, are most of the way to being an eval already. Fourth, explicit non-goals — what the system is deliberately not trying to do. A claims-flagging tool that explicitly does not make approve/deny decisions, only flags for human review, needs that boundary stated as forcefully as its capabilities, because agents (and engineers) will happily generalize a capability past its intended scope if nothing stops them. Fifth, and most often skipped, decision points resolved in advance — the specific list of judgment calls the spec author identified and made, rather than leaving to whoever builds it. This is different from edge cases; edge cases are about unusual inputs, decision points are about ambiguous design choices even in the ordinary path — does "suspicious" mean flagged for human review or auto-denied, does a tie in confidence scoring default to flag or default to clear, does the system version its rules and how.

A toolkit for finding your unknowns

The hard part of hardening a spec is rarely the writing — it's knowing what you don't know. Most spec gaps are invisible to their author precisely because the author's own mental model fills them in automatically, without noticing there was a gap to fill. The following five techniques are ways of exposing those gaps before they become production incidents, and they work because each one forces a different kind of surprise.

1. Blind-spot pass

Give the model your current spec and ask it directly: "What does this spec fail to specify that a competent engineer would need to know before building this?" This works because a model reading cold, without your implicit context, will surface exactly the assumptions you're too close to see. Push it past the first answer — the first pass tends to be generic ("add more error handling"); the second and third passes, prompted with "go deeper, get specific to this domain," surface the real gaps.

2. Divergent prototypes

Generate two or three independent implementations from the same spec — different sessions, ideally different models — and diff the decisions each one made differently. Where they agree, the spec was clear enough that clarity wasn't the issue. Where they diverge — one auto-denies at high confidence, another always routes to human review, a third treats missing data as low-risk while another treats it as high-risk — every divergence is a decision point your spec left unresolved. This is one of the highest-signal techniques in the toolkit because it doesn't rely on anyone, human or model, correctly guessing what's ambiguous; the ambiguity announces itself.

3. Interview-me

Instead of writing the spec and then checking it, hand the model your rough one-line request and ask it to interview you before writing anything. A well-prompted interview pass will ask the boundary questions a rushed spec author skips — what should happen on conflicting data, who is the user when there are multiple stakeholders, what's the cost of a false positive versus a false negative. This front-loads the discovery that would otherwise happen expensively, mid-build.

4. Quiz-me-back

The inverse of interview-me: after you've written a spec, ask the model to generate a quiz about it — questions a builder would need to answer correctly to implement it right — and then try to pass your own quiz. Questions you can't answer confidently point directly at unresolved decision points. This technique is cheap, fast, and catches the specific failure mode where a spec author thinks they've been thorough because the prose reads smoothly, when smooth prose and resolved ambiguity are unrelated properties.

5. Reference-as-map

When a similar system already exists — a competitor's product, a legacy internal tool, an industry-standard workflow — give it to the model as a description of the territory rather than trying to spec the destination from scratch. "Here's how our current manual claims-review process works end to end; the new system should replicate the reviewer's actual decision logic, not a simplified version of it" gives the model something concrete to map against, and concrete maps surface gaps that abstract descriptions don't.

The spec for agent systems: policies, examples, and the eval set

Everything above generalizes across software, but agent systems have a specific version of the spec that's worth naming directly, because it's the one this course will keep returning to. For an agent, the spec is not a single document — it's three things working together: the policies (the rules and constraints, analogous to the non-goals and acceptance criteria above), the worked examples (behavioral examples, but specifically chosen to cover the decision points and edge cases that matter for this domain), and the eval set. The eval set deserves special emphasis, and lesson 12 will develop it fully, but the claim to plant here is this: the eval set is the executable part of the spec. A policy document can be misread; a worked example can be memorized instead of generalized; but an eval set — a battery of concrete cases with known-correct outcomes — is the one part of the spec that can actually be run, scored, and used to verify that a build (whether written by a human or generated by an agent) satisfies the spec's intent rather than just its wording. If you've written acceptance criteria as described above, you are most of the way to an eval set already; the remaining work is making those criteria runnable rather than just readable.

// The three-part spec for an agent system, sketched

policies:
  - "Flag claims for human review; never auto-deny."
  - "Confidence tiers: A (auto-flag), B (flag with note), C (log only, no flag)."
  - "Non-goal: this system does not determine claim payout amounts."

worked_examples:
  - input: "Roof leak claim, 3 days post-storm, no neighboring claims in 72h window"
    expected: { action: "flag", tier: "B", reason: "timeline inconsistency" }
  - input: "Water damage claim, same-day as storm, 12 neighboring claims same ZIP"
    expected: { action: "log_only", tier: "C", reason: "consistent with storm pattern" }

eval_set:
  - id: "eval-014"
    input: "Claim with missing prior-history data, otherwise clean timeline"
    expected: { action: "log_only", tier: "C" }   // resolved decision point: missing data != high risk
    # this case exists BECAUSE divergent prototypes disagreed on it

Why this matters more as autonomy increases

The asymmetry between easy and hard specs gets sharper, not flatter, as you give agents more autonomy and longer task horizons. A human developer building from a vague spec accumulates small corrections through code review, standups, and Slack threads — the ambiguity gets resolved through friction, slowly, but it gets resolved. An autonomous agent running a multi-step task from a vague spec accumulates compounding wrong assumptions, because each step builds on the last one's unstated interpretation, and nothing forces a pause to check. By the time a human reviews the output, the agent hasn't made one ambiguous decision — it's made twelve, consistently, in a direction the spec author never intended and would object to if asked directly. This is precisely why the toolkit above front-loads the cost of finding decision points before the build starts, rather than relying on review to catch them after. Review catches wrong code. It catches unspecified-and-therefore-inconsistent behavior far less reliably, because inconsistent-but-plausible output is exactly what a competent reviewer, skimming for correctness rather than auditing for spec coverage, will wave through.