The Model Layer: Commodity Intelligence and Capability Overhang

Reading time ~12 min · Quiz at the end

Every other lesson in this course tells you to spend less attention on the model and more on the harness. This lesson is the exception — not because the doctrine changes, but because making good harness decisions requires understanding one specific, slightly strange fact about how models acquire capability. If you don't understand this, you'll either overspend on frontier models for tasks that don't need them, or underspend on the tasks that genuinely do, and you'll make both mistakes with total confidence because the mistake doesn't announce itself.

Models are grown, not designed

A traditional software capability is designed: an engineer decides the system should support CSV export, writes the code, and it works or it doesn't, predictably, the same way every time. A model capability is grown: it emerges from scaling up training compute, data, and parameters, and nobody on the training team decided in advance "at this scale, the model will become reliably good at extracting structured line items from a messy invoice PDF." It just starts happening, sometime between one training run and the next, and the people who trained the model often find out about a new capability from users after release, not from their own internal evals beforehand. This produces a phenomenon researchers call capability overhang: at any given moment, a released model has abilities that haven't been discovered yet, sitting latent, waiting for someone to find the right way to elicit them. A model might fail a task badly when you ask it directly in chat, and then succeed at the identical task once you let it write and execute code instead of reasoning in prose — because "compute this by writing a script" turns out to route around exactly the kind of arithmetic or multi-step tracking error the model makes when forced to hold state in its own generated text. The capability was there all along; it just needed a different harness to surface it.

Model tiers and selection economics

Because capability is uneven and unpredictable in its arrival, but broadly correlated with model size and training cost, you can still make useful generalizations about tiers — as long as you re-verify them for each new model generation, which the last section of this lesson insists on. The organizing principle: model capability is best understood as how underspecified a task can be while the model still lands on the right outcome. A cheap model needs the task fully specified, with obvious failure modes, and does that well and fast. A frontier model can be handed a genuinely ambiguous problem — "figure out why churn spiked in this segment" — and do the work of figuring out what the task even is.

Where frontier value concentrates: the two ends of a project

A common mistake is routing every step of a project through the most expensive model available, on the theory that more intelligence can only help. In practice, the marginal value of frontier capability is not evenly distributed across a project's lifecycle — it concentrates at two ends. At the start, frontier models earn their cost doing the work that cheap and mid-tier models cannot do at all: taking an underspecified business ask — "reduce claims processing time" — and turning it into an actual scoped plan, identifying what data exists, what's missing, what the edge cases probably are, and what a first version should and shouldn't attempt. This is exactly the kind of task where the model needs to operate without a clear spec, because producing the spec is the task. In the middle, once the spec exists, the work is usually mechanical relative to that spec: apply the extraction rules to ten thousand documents, generate the first-draft summary for every claim in the queue, produce copy variants for every SKU in the catalog. This is bulk execution against a clear target, and it's where a mid-tier model, run at volume, delivers the best cost-adjusted throughput. Running frontier models for this stage is usually paying a premium for capability the task doesn't need. At the end, frontier value reappears for the adversarial audit: reviewing a sample — or, cost permitting, all — of the mid-tier model's output looking specifically for the plausible-but-wrong cases that a rubric-following review might miss. This is a different cognitive task from bulk generation — it's closer to a skeptical domain expert reading work with the specific intent of finding what's subtly off, which rewards exactly the kind of general judgment frontier models are best at.

Project: Automate first-pass drafting of claim denial letters

Stage 1 — Scoping (frontier model)
  Input: vague mandate — "speed up denial letter turnaround"
  Output: scoped spec — categories of denial, required legal
  language per category, escalation triggers, edge cases to exclude

Stage 2 — Bulk drafting (mid-tier model)
  Input: scoped spec + 4,000 claims/month
  Output: first-draft letters against the spec, high volume,
  low per-unit cost

Stage 3 — Adversarial audit (frontier model, sampled)
  Input: sample of drafted letters + spec
  Output: flags on letters that look compliant but subtly
  misapply a denial category or omit required language

The acceptance-test corollary

A further consequence of thinking in terms of specified-versus-ambiguous tasks: an acceptance-tested cheap model often beats an untested strong one. If you can write a deterministic check for correctness — the extracted policy number matches a checksum, the categorization falls into one of five valid buckets, the generated JSON validates against a schema — then you've converted part of the judgment burden from "trust the model's reasoning" to "verify the model's output," and verification is a task cheap models and plain code are extremely good at. A well-specified task with a tight acceptance test, run on a cheap model with automatic retry on failure, will often be more reliable in aggregate than the same task run once on a frontier model with no verification step at all, because the frontier model's occasional confident mistake sails through unexamined while the cheap model's frequent obvious mistakes get caught and retried automatically.

Everything is perishable — re-test every generation

The tier table above will be wrong in places within a year, possibly within months, because the entire field moves on a scaling and post-training curve that keeps shifting where each tier's boundary sits. A task that requires frontier judgment today may be handleable by a mid-tier model after the next round of post-training improvements, particularly the kind of task that turns out to benefit from a specific tool or scaffolding change rather than raw scale — echoing the capability overhang point from the start of this lesson. The inverse also happens: a task you've been comfortably running on a cheap model can start failing in new ways after a vendor updates that model's weights behind the same API endpoint, silently changing behavior you had implicitly relied on. Two disciplines follow directly from this. First, treat model-tier assignments as a living decision, not a one-time architecture choice — put them in your documentation with a review date, not just a design doc that gets filed away. Second, build the eval suite (lesson 12) that lets you actually re-test a tier assignment in an afternoon instead of a quarter. Without an eval suite, "should we downgrade this task to a cheaper model now that the new version shipped" is a question nobody has time to answer, so the default answer becomes "leave it as is," which is how organizations end up paying frontier prices for tasks a cheap model has handled comfortably for six months.

Treat model selection as a portfolio, not a marriage

The practical implication of everything above is that a mature harness rarely runs on one model. A single enterprise workflow — the claims agent from lesson 2, say — plausibly uses a cheap model to classify inbound documents, a mid-tier model to draft the adjuster's summary, and a frontier model for the periodic audit of a sample of decisions and for scoping any new claim type the business adds. Each assignment is a decision made against the specific task's ambiguity and stakes, revisited on a cadence, and swappable independently of the others because the harness — not the model choice — is what encodes the actual logic of the system. This is also a risk-management position, not just a cost one. A portfolio of models across tiers and potentially across vendors means no single vendor's price increase, rate limit, deprecation notice, or outage takes down your entire agent estate at once. Vendors compete hardest exactly where you have real alternatives, so maintaining genuine switchability — proven by actually running eval suites against more than one vendor periodically, not just believing you could switch — keeps your negotiating position and your production resilience aligned.

What the enterprise-scale evidence shows

The tier framework above is easy to mistake for opinion until someone measures it at scale, so it is worth pointing to the largest public test of these exact claims. Databricks (2026) built an internal benchmark from its own engineers' real coding tasks against a multi-million-line, ten-plus-language codebase — Scala, Go, Rust, Java, Python, TypeScript, Bazel, Protobuf — and evaluated model-and-harness combinations on both cost and quality. The models did not spread out along a continuum; they clustered into three rough capability tiers, the same shape this lesson draws from first principles. Top-tier models solved essentially everything but cost the most; the medium and lower tiers stayed highly effective on common tasks — flag flips, config updates — at a fraction of the price. That is the specified-versus-ambiguous axis showing up in real telemetry rather than a whiteboard argument.

The commodity thesis showed up too, and more sharply than the "revisit your vendor" framing above might suggest. The cost-quality Pareto frontier in their study was covered by no single vendor — it needed a mix of models from OpenAI, Anthropic, and open source to trace out, which is the portfolio argument stated as an empirical result rather than a risk-management preference. The open model GLM 5.2 landed in the top capability tier, statistically tied with Claude Opus 4.8 on quality, while costing $1.28 per task against Opus's $1.94 — roughly two-thirds the price for indistinguishable quality. If you needed a single data point for why brand loyalty is a tax rather than a strategy, that is it: the frontier is now a moving set of interchangeable options, and an open model can sit inside it.

Two subtler findings sharpen the discipline this lesson asks for. First, Databricks' AI-gateway telemetry showed that about a quarter of their engineers' real agent tasks were low-complexity and roughly sixty percent medium-complexity — yet engineers reached for the most expensive model for nearly all of it. This is capability overhang's economic twin: where overhang is latent capability nobody has elicited yet, this is latent savings nobody has captured yet, sitting in plain sight because the expensive default never announces itself as waste. Second, and most important for how you actually pick a tier: per-token price turned out to be a poor predictor of per-task cost, because a cheaper-per-token model that reads more and works longer can cost more per finished task. The consequence is direct — you cannot read the right tier assignment off a vendor's pricing page. You have to benchmark candidate models on a sample of your own workload, which is precisely what the eval suite from lesson 12 is for.