The OS Fallacy: Why Stronger Models Make Orchestration Matter More, Not Less
Also in Chinese: 中文版
An argument I keep running into
It goes roughly like this. The engineering layer around agentic coding — context trimming, orchestration frameworks, external rule systems — exists only to route around the model’s current weaknesses. Once reasoning models get good enough and long context gets cheap enough, none of it has a reason to exist. So don’t invest in orchestration; it’s a transitional crutch.
Half of that is right, and the half that’s right is sharp. What I want to pin down is where the other half goes wrong — because it isn’t wrong about direction, it’s wrong about level.
It is the same claim as: because CPUs keep getting faster, the operating system will disappear.
Faster CPUs really did kill a set of OS-layer things. Memory overlays. Hand-managed DMA. Assembly inner loops written to save a few cycles. Those were core craft once; nobody touches them now. But process management, permission control, device drivers, filesystems — those didn’t disappear. They got more important as the hardware got stronger, because stronger hardware means more things running on it, which means more boundaries to isolate, schedule, and audit.
First, the half that’s right
Underneath the argument is a principle I fully accept:
Any solution that relies on a middle-layer abstraction to paper over a deficiency in the layer below will eventually be obsoleted by progress in that lower layer.
It’s a good principle, and it has already killed things. Not speculatively — with numbers:
| Obsoleted | Replaced by | Evidence |
|---|---|---|
| CoT prompting | Reasoning built into the model | Wharton GAIL: on reasoning models, CoT buys +2.9–3.1% accuracy for +20–80% time |
| Output parsers / retry loops | Structured Outputs | Compliance 35% → 100% |
| Few-shot template machinery | Zero-shot capability | On some tasks few-shot actively hurts |
| Basic tool-calling orchestration | Native function calling | Now a standard parameter in every major API |
| Vision OCR pipelines | Native multimodal input | Frontier models eat images directly |
If most of your 2023 agent code was this, that code is gone. That’s the principle earning its keep: it tells you in advance which work will be short-lived.
The error happens at the next step — classifying all orchestration as “papering over model deficiency.”
A methodological aside: the argument cites a paper that doesn’t exist
The third pillar of the case is that a new generation of sparse attention (written as “NSA+”) lets models manage their own context, making external context management unnecessary.
I went looking. There is no “NSA+” in the published literature. What exists is NSA (February 2025, arXiv 2502.11089, ACL 2025 best paper) and DSA (DeepSeek Sparse Attention, shipped in V3.2 in September 2025), plus a later paper called NOSA (October 2025). No “NSA+”.
And the NSA paper is explicit that it is a compute-efficiency optimization, not a comprehension improvement:
Trainable sparse attention can only alleviate the memory-bound bottleneck; it does not reduce KV cache size.
In its sparse phase each query selects roughly 2048 KV entries. That is approximation, not a guarantee that the model “understood” the whole context.
Meanwhile the evidence runs the other way. Chroma tested 18 frontier models and found that every model, at every input-length increment tested, produced worse output as tokens were added; models advertising 200K windows became unreliable around 130K. “Lost in the Middle” persists, with 30%+ accuracy drops in the middle of a long input. The gap between effective and advertised context runs as high as 99%.
So the premise — long context will absorb context management — hasn’t held, and the citation propping it up is fictional.
I’m calling this out not to score a point on an author. It’s because this genre of claim is now trivially easy to generate with AI assistance, and what comes out looks exactly right: the shape of the terminology is right, the shape of the timeline is right, and the confidence is often higher than a real paper’s. The only defense is going back to the primary source once. That’s a sub-argument of this post in its own right.
So where did orchestration actually go: three-way redistribution
It didn’t disappear. It moved, and the total amount went up.
One — absorbed into model weights via RL. Cognition’s SWE-1.5 and Cursor’s Composer 2 are both end-to-end RL in real task environments, baking plan-execute loops, error recovery and tool selection into the weights. This is the strongest indirect confirmation of the original argument — except the mechanism is the opposite of what it predicted. Not “remove the scaffold,” but scaffold → training data → model weights. The scaffold wasn’t deleted. It was eaten.
Two — standardized into protocol. MCP collapses N×M tool integration into N+M. SDK downloads went from 2M to 68M per month; it joined the Linux Foundation. That isn’t orchestration disappearing, it’s orchestration becoming infrastructure — the way HTTP did for the web. Nobody says “I do HTTP orchestration” anymore, because it became ground.
Three — thinned into a minimal harness. Claude Code’s agent loop is roughly 50 lines (while(tool_call) → execute → feed results → repeat), 18 tools, single-threaded, no multi-agent swarm. And it sits inside 29,000+ lines of harness. The heart got thinner; the skeleton got thicker.
A testable criterion
Break those 29,000 lines apart and you get seven categories. Each has a structural reason the model can’t absorb it:
| Category | Contents | Why it stays in the harness |
|---|---|---|
| A. Interface to the outside world | Bash, file I/O, Git, MCP, IDE | Physical gap: models emit text, not syscalls |
| B. Trust boundary | Permissions, approval, sandbox, allow/deny lists | Architectural: the audited party cannot be the auditor |
| C. Cross-session state | Session persistence, task tracking, project memory | Time dimension: state must outlive any single context |
| D. Multi-agent coordination | Sub-agent dispatch, worktree isolation, message routing | A distributed-systems problem, orthogonal to model capability |
| E. Economics | Token budget, prompt caching, model routing | The model can’t see global cost |
| F. Human interface | Terminal rendering, diffs, hooks, slash commands | An interface outside the agent, not a reasoning problem |
| G. Engineering scaffolding | Schema validation, tool search, retries | Absorbable in theory; not yet worth it |
A through F all involve something outside the model: physics, trust, time, coordination, economics, people. G is the only category that gets eaten as costs fall.
Which compresses to one line:
Capability that can be compressed into “better next-token-prediction” gets absorbed by the model.
Capability that touches the world outside the model (physics / trust / time / coordination / economics / humans) stays in the harness.
Run it forwards or backwards. Backwards is the more useful direction: if what you’re building can be compressed into “a better prompting pattern,” the next model generation will probably delete it. Auditing your own technical bets with that question beats reading ten trend forecasts.
But does scaffold still matter? That part is falsifiable
No need to argue. Look at the numbers:
- Same model, different scaffold: a 12–30 point spread. On SWE-bench Pro the swing between a basic and an optimized scaffold exceeds 22 points; better context retrieval alone is worth 4–10.
- Epoch AI’s finding: a great model with a mediocre scaffold loses to a good model with an excellent scaffold.
- On SWE-bench Verified the top of the board is 85–94%. On SWE-bench Pro — harder, closer to real work — the top score is about 23%.
That last pair deserves a pause. Same models, a task set that looks more like real work, and the score falls to a quarter. What fell away is not a gap in model intelligence; it’s a gap in how well the world was assembled around it — which is exactly the harness’s job. Anthropic’s own engineering blog states it most precisely:
the principal deficit of agentic systems is not the model’s intelligence but the quality of the world assembled for it.
And yes, “simple beats complex” has authoritative backing — Anthropic’s official guidance says plainly that the most successful implementations didn’t use complex frameworks, but simple composable patterns. But “simple” and “absent” are different things. Claude Code’s 50-line loop gets to be simple precisely because 29,000 lines sit underneath it.
Two judgements I changed as a result
One: ease of use is a debt paid by the engineers below you.
The simpler the upper abstraction, the more complex the layer beneath — they are inversely coupled, not independent. One line of SQL sits on a PhD-grade query optimizer. pip install sits on HTTP/2 + TLS + DNS + TCP + IP. git push sits on distributed consistency and conflict detection. The iPhone’s “It Just Works” sits on iOS plus a Secure Enclave plus thousands of engineers.
Four mechanisms drive it: more users means wilder requirement dimensions; every added abstraction adds another “is it A, is it B, or is it the interface” search space; once an upper interface stabilizes it can’t move, so every change below must disguise itself as “the interface didn’t change”; and the more foundational something is, the more catastrophic its failure, so the reliability bar only ratchets up.
The counterintuitive conclusion: if your software feels simple to use, it isn’t because the problem was simple. It’s because somebody absorbed the complexity for you.
Two: cheap is a lagging indicator; hard is a leading one.
CRUD is extraordinarily cheap today. Not because it’s simple, but because forty years of infrastructure investment got compressed into a pip install — you are burning a fossil fuel of congealed complexity.
| Stage | Difficulty | Era |
|---|---|---|
| Frontier | PhD thesis | Codd’s relational algebra, 1970s |
| Framework | Expert domain | SQL standardization, 1980s |
| Standard | Usable by most | MySQL / Postgres, 1990s |
| Infrastructure | Invisible | Rails / Django, 2000s |
| Commodity | Assembled in hours | CRUD, 2020s |
Harness work is hard today, which means it sits on the first row of that table. Today’s hard is 2040’s pip install.
So: betting on what’s cheap means consuming past complexity, with declining marginal value. Betting on what’s hard means producing future complexity — rising marginal value, higher variance.
The SRE version of this
I work on infrastructure reliability, so there’s a blunter way to say it from where I sit.
Interface problems do not disappear because one side of the interface got stronger. Tool integration, safety gates, system-boundary management, cross-session error recovery — none of these paper over a model deficiency. They manage the seam between the model and the outside world. A stronger model only makes the actions on the far side of that seam more forceful and more consequential, which makes the seam itself more worth designing.
That is precisely the operating system’s history. Faster CPUs didn’t remove the OS; they made “who may touch which memory, which process may issue which syscall, who rolls back when it fails” matter more.
Agent harnesses are on the same path. With more urgency than the OS had — because a CPU doesn’t confidently fabricate an answer that doesn’t exist, and a model does.
Every figure and paper here is from a public source: Wharton GAIL on CoT, Chroma’s context-rot report, Epoch AI on scaffold variance, Scale AI’s SWE-bench Pro leaderboard, NSA (arXiv 2502.11089), a16z’s LLMflation data, and Anthropic’s engineering blog and official agent guidance. Checkable — arguments welcome.