Developers who regularly use AI for coding, information retrieval, or building workflows may have noticed that Cognition, the team behind Devin, and Anthropic seem to hold opposite views on multi-agent systems. In June 2025, Cognition published “Don’t Build Multi-Agents”, advising against building multi-agent systems. Around the same time, Anthropic published “How we built our multi-agent research system”, explaining why they chose a multi-agent design for their research product. Then in April 2026, Cognition followed up with “Multi-Agents: What’s Actually Working,” noting that they had also started running multi-agent systems in production, though only in specific scenarios. On the surface, the two camps look like they are contradicting each other, but they are actually describing the same reality. Once you see why, you can choose an architecture without burning several times the tokens for nothing—or hesitating to parallelize when you actually should.
The two teams only sound at odds because they are looking at opposite ends of the same trade: Anthropic is looking at what the extra compute buys, while Cognition is looking at the ballooning API bills and the communication overhead between roles. Multi-agent systems do not make a model any smarter on a single reasoning pass; they use orchestration to throw more tokens at the same problem.
Anthropic’s June 2025 production retrospective on its multi-agent research system broke down that spending in detail. When a single agent calls tools repeatedly across multiple turns, it uses roughly 4x as many tokens as a standard chat interaction; a multi-agent system uses around 15x. On complex web-browsing benchmarks like BrowseComp, how many tokens a system spends maps very directly to its final score.
What does 15x the tokens buy? When Anthropic plotted BrowseComp scores against token usage across systems, they found that token usage alone accounted for 80% of the variance in scores: systems that spent more tokens generally scored higher. Token spend may not be the only factor, but in this dataset it is the single variable that best explains who wins. As Anthropic’s engineers put it, multi-agent systems work mainly because they “help spend enough tokens” to solve the problem.
Multi-agent is not magic that improves every engineering metric across the board. It is an explicit trade: you pay several times the tokens, waiting time, and coordination overhead to buy one or two specific metrics, while other metrics stay flat or even get worse. Anthropic itself simply describes this setup as an orchestrator-worker workflow rather than a brand-new architecture.
Before upgrading a single agent into a multi-agent team, people usually expect four payoffs: stronger results, faster execution, end-to-end coverage across the workflow, and hands-off mutual supervision. All four sound like free wins, but each comes with a steep price tag. Worse, with a default setup, you often pay three to five times the cost without getting the benefit you wanted.
Intuitively, you might expect several models discussing and dividing up the work to write better code than one model working alone. A realistic test of software development is to hand a model a complete product specification and ask it to build a full-featured web application from scratch. Vals AI’s evaluation uses Vibe Code Bench, a benchmark of 100 such tasks. Taking half of those tasks, they tested GPT-6’s high-tier Sol and Claude Opus 5.5 at both medium and max reasoning settings, comparing solo runs against multi-agent teams across eight configurations in total. Each model ran in its own harness, with a single run per configuration, comparing score differences across tasks.
When the results came back, running a team cost 1.8x to 5.1x as much as running solo. Yet out of the four comparisons, only Sol at the medium setting achieved a statistically significant score gain from teaming up (+7.3 points); the other three showed no statistically significant difference from working solo.
If you instead save the extra budget spent on a team and use it to turn a single Sol model’s reasoning setting up from medium to max, the score jumps by 11.4 points—more than the gain from forming a team at medium. The Opus max team did post the highest overall score, but it cost 30x as much as Opus medium solo while scoring only 1.6 points higher, a difference that was also not statistically significant. What ate the budget was repeated cached-input reads: every time a sub-agent stepped in to work, it had to read the product spec and current workspace state from the top, getting billed on every call. Even though cached reads are cheap per token, resending the context on every call across every sub-agent adds up fast once call counts climb.
Another natural assumption is that many hands make light work: multiple agents working together should finish faster than one. In Vals AI’s experiments, however, only the Sol teams were actually faster; the Opus teams dispatched subtasks in serial waves and, at their slowest, took 2.3x as long as a single agent.
As Anthropic noted in its production retrospective, software coding tasks are far less parallelizable than information retrieval and research tasks. Once a task requires all agents to share the same context, or its steps are tightly coupled, a multi-agent setup wastes huge amounts of time waiting and synchronizing, making it a poor fit today.
Once you have a team, it is tempting to hand the entire pipeline—exploration and planning, code implementation, and test verification—over to sub-agents. Another study on building software from scratch tested this directly, giving models a prompt and a set of test cases and asking them to write a program from the ground up. Cohere’s team-scaling study used ProgramBench, a benchmark of 200 such tasks; each task provides an original binary that the agent can execute but cannot inspect as source code, and asks the agent to rewrite a behaviorally matching version from scratch. Every configuration was run three times and averaged to reduce single-run noise.
Well-organized teams beat solo agents on ProgramBench by up to about 10 percentage points, at 3x to 7x the runtime cost. But an even sharper comparison came out of the same study: delegating only exploration and planning to sub-agents achieved nearly the same score as delegating planning, implementation, and verification altogether, while cutting the bill in half.
Exploration and planning here do not mean vague brainstorming. Planning workers repeatedly run the reference program to probe its interfaces, edge cases, and behavior, then compile their findings for the lead agent; the lead takes that map and writes the implementation and runs verification itself. In Cohere’s numbers, an asynchronous team that delegated only planning scored 63.6%, virtually tied with full-pipeline delegation, while dropping the cost from $510 to $249. The reason is straightforward: on this benchmark, the bottleneck is figuring out what to build, so putting workers on that bottleneck is what pays off. Cohere also contrasted this with another benchmark, Commit0, where specifications, documentation, and unit tests are already provided in full; there, the bottleneck shifts to getting the implementation right, and the most effective stage to delegate shifts to implementation accordingly. If you outsource every stage to sub-agents indiscriminately without asking where the task is actually stuck, roughly half of your extra spend is wasted.
Fourth, many people assume that setting up several roles to check each other—one writing code, one running tests, one reviewing—will make the system self-correcting. Yet once you actually run multiple agents in a relay, you routinely see two roles redoing the exact same work, claiming they modified the code without touching a single line, or declaring victory before verification ever passes. Researchers collected more than 1,600 execution traces across seven popular multi-agent frameworks and organized the breakdowns into MAST, a taxonomy of 14 failure modes across three major categories.
According to MAST, step repetition accounts for 15.7% of all failures, and mismatches between reasoning and actual action account for 13.2%. Among the frameworks analyzed was the well-known early system ChatDev; without changing the underlying model, simply clarifying role boundaries noticeably raised success rates. Dropping more roles into a system does not automatically make it more reliable—when responsibilities are blurry, it manufactures coordination failures a single agent would never have made.
Why, then, are experienced engineering teams still willing to pay for multi-agent systems in certain cases? Because spending several times the budget can genuinely buy four concrete engineering metrics. Each one, however, comes with strict preconditions—the investment only pays off when your task actually meets them.
Only when a task naturally splits into mutually independent subtasks—and does not degenerate into serial waiting at runtime—can a multi-agent system deliver shorter end-to-end wall-clock time. On the Dwarkesh Podcast in September 2026, Noam Brown offered a simple rule of thumb: four agents in parallel cut end-to-end time in half while doubling the price, with diminishing returns as you add more parallelism. He also stressed the precondition: that trade-off only holds when the work is genuinely parallelizable. For a task like writing a novel, where foreshadowing and tone must fit tightly together from start to finish, throwing more agents at it will not make it any faster.
The second metric you can buy is thorough coverage: catching what a single agent misses when a task calls for breadth-first investigation across many leads, when the relevant information cannot fit inside a single context window, when dozens of complex tools must be called at once, or when a product spec is so sprawling that a solo agent drops pieces of it. As Anthropic noted in its multi-agent retrospective, on tasks with a wide surface area, multiple agents can cast a wide search net in parallel.
In large-scale application development, this advantage shows up as independent verification of missed details. Looking back at Vals AI’s Vibe Code Bench experiment, in the single configuration where a team achieved a statistically significant gain, nearly all of that gain came from hard applications: teams beat solo runs by roughly 15 points on hard apps, versus roughly 1 point on easy ones. Inspecting the execution traces shows why: faced with a sprawling product spec, a solo agent frequently claims in its report that it tested everything while actually skipping several features or test checks. Multi-agent teams pull ahead on hard tasks because dedicated verification roles catch the gaps that a solo agent reports as done but never actually built, checking them off item by item.
The third capability money can buy is search diversity, aimed specifically at getting unstuck on ultra-long-horizon tasks. It applies when a task runs for hours or days, an early wrong turn will send the agent down a dead end, and the system actively flushes the parent context after branching out to explore. When a single agent works on a complex problem for hours on end, once it commits to an initial technical approach, it tends to circle inside the same mental rut and rarely escapes on its own. Microsoft Research calls this trap an “idea basin” and designed Fork-and-Flush to break out of it: at key decision points, the system spawns multiple branches like an OS process fork to probe different approaches (Fork), and once a promising branch emerges, it wipes the parent’s long reasoning history clean (Flush), passing along only a clean interim conclusion.
Microsoft Research tested this mechanism across a suite of long-horizon tasks lasting from several hours to several days per run, using GPT-5.6-terra on 11 tasks and GPT-5.5 on two tasks (kernel and tinyCIFAR), executed through the GitHub Copilot CLI. Holding total compute constant, Fork-and-Flush achieved an aggregate score of 0.78—far ahead of a single long run at 0.47 and also well above running four independent trials and picking the best, a 66% relative improvement over a single run. Ablation studies measured each step’s contribution separately: flushing history alone or forking paths alone brought modest local gains, but only combining the two delivered the full improvement. Forking selects a better route across different idea basins, while flushing cuts off the inertia of the parent’s reasoning history so the chosen branch does not slide back into the old rut.
The fourth payoff is context isolation. After a lead agent has spent too long wading through tens of thousands of lines of code or error logs, its window is cluttered with dead-end hypotheses and trial-and-error residue, scattering its attention and skewing its judgment. Chroma’s July 2025 study on Context Rot confirmed a general rule: even on very simple tasks, as you feed a model longer and longer input text, its performance steadily degrades.
Once the main agent has slogged through tens of thousands of lines of code and stack traces, its context is saturated with the residue of failed hypotheses. Paying to spawn a sub-agent at that point is really buying an uncontaminated context window. In its April 2026 engineering post on multi-agent systems, Cognition shared a familiar observation from practice: a review agent with a clean context routinely spots bugs in the code that the authoring agent stared at for ages and missed.
Even after you sort out which metrics money cannot buy and which ones only pay off under specific conditions, putting multi-agent systems into practice still runs into three common traps: capability ceilings, mechanism choices, and read/write organization. If you get these three wrong, no amount of parameter tuning will save the system.
On capability, a common hope is that you can team up several weaker models and let the wisdom of the crowd produce the reasoning power of a frontier model. Early studies such as More Agents Is All You Need did show that when you sample and vote across multiple agents, output quality rises with the number of instantiated agents, scales with task difficulty, and can be combined with more elaborate collaboration methods.
Still, that approach cannot cross the capability ceiling of the lead model itself. Cohere’s cross-model comparisons on ProgramBench provide a direct counterexample: whether a team delivers any gain depends mainly on the strength of the lead agent. With the earlier Opus 4.7 as lead, the model almost never delegated tasks and gained nothing from having a team; certain versions of Haiku and Gemini delegated eagerly, yet also produced almost no team gain; only with the newer Opus 4.8 and GPT-5.6 as lead did the system delegate effectively and actually benefit. Pairing a weaker lead model with a crew of cheap workers scored worse than a single strong model working solo. Whether you use sampling-and-voting or orchestrator-worker delegation, what you are buying is higher sampling hit rates and broader planning coverage at the same capability tier—not a generational leap in base reasoning ability.
On mechanism, there are two common assumptions: first, that a complex task requires an autonomous agent running open-ended multi-turn exploration loops; second, that on long-running tasks, periodically summarizing a long conversation history works just as well as flushing the context. On the SWE-bench Lite bug-fixing benchmark, the minimalist system Agentless directly refuted the first assumption: fixing repository bugs does not necessarily require letting an agent wander around on its own. Breaking the process into a fixed three-stage pipeline—localization, repair, and patch validation—achieved a 32% solve rate at just $0.70 per issue, beating every open-source software agent at the time.
As for the second assumption, Microsoft Research’s Fork-and-Flush experiments refuted it as well: even when conversation history is periodically compressed into summaries, the agent stays trapped in the same idea basin. Compression merely rewrites old reasoning in shorter form and leaves it sitting in the window; flushing throws that baggage out completely. Open-ended autonomous loops buy dynamic pathfinding on ill-defined problems, and fork-and-flush buys a clean break from the parent’s reasoning inertia. When the workflow is fixed, a fixed pipeline is much cheaper; when reasoning is stuck in a rut, summarizing the history will not pull it out.
On organization, a popular vision is a decentralized swarm where peer agents talk freely and edit the same codebase or document at the same time. Cognition’s engineering retrospective poured cold water on that idea: in real engineering, unstructured swarms mostly generate noise. In the multi-agent systems that actually work today, writes remain single-threaded; extra agents contribute judgment without directly mutating shared state, and the pattern that holds up in production is dispatch, aggregation, and central control.
LangChain’s guide to multi-agent systems explains why from the basic asymmetry between reads and writes: reads parallelize naturally, whereas parallel writes require both context synchronization and output merging, where the slightest misalignment produces irreconcilable conflicts. That is also why Anthropic’s multi-agent research system only sends sub-agents out to read and gather material in parallel, while keeping final synthesis and writing inside a single lead agent call. Organizationally, what multi-agent really buys is parallel throughput for reading, retrieval, and review—modifying state and writing output must stay single-threaded.
All of the experiments and engineering lessons above can be compressed into a four-row reference table. Whenever you are debating whether to add another agent—or how to spend budget to buy better performance—take ten seconds to see which row matches your current bottleneck.
| Where the Task Is Stuck | Does Multi-Agent Help? | More Cost-Effective Move |
|---|---|---|
| Single-step reasoning is too hard, steps are tightly coupled, or multiple agents edit the same place | No | Turn up single-agent reasoning effort, or use a fixed pipeline |
| Subtasks are independent and you just want shorter wall-clock time | Yes | Verify it won’t degenerate into serial waves, then dispatch in parallel |
| The search surface is too wide, or complex specs lead to missed checks | Yes | Delegate only exploration and planning; add independent verification for missed checks |
| Long runs get stuck in a dead end, or the context window is cluttered | Yes | Fork and flush history; spawn a clean window for review |
Once you confirm that your task falls into the bottom three rows and genuinely needs a team, the next question is how to keep the team’s bill down. Capability routing does not buy a fifth benefit here. It is a cost reducer after you have already decided to run a team: it changes how much extra you pay, not the fact that a team costs more than running solo.
Back to those four hard metrics: as long as a task lets you separate decision-making from scouting, and your lead model is strong enough, a multi-agent architecture lets you mix model tiers to lower the bill inside the team. Cohere’s heterogeneous-team experiments on ProgramBench measured this directly using another generation, the GPT-5.6 family, where Sol is the high tier and Luna is the low tier. A single Sol lead paired with sixteen much cheaper Luna workers for exploration and planning scored 61.8%, nearly matching an all-Sol team while costing just $155.
It is worth keeping the accounting straight here. When delegating only exploration and planning, an all-Sol team costs $249; swapping the planning workers to Luna under a Sol lead is what brings the bill down to $155. Comparing $155 against the $510 all-Sol full-pipeline team actually conflates two different savings: not delegating implementation and verification, and using a cheaper model for planning. And $155 is still the bill for a 17-agent team—the solo baseline is not even in that comparison. Capability routing, in other words, does not prove that multi-agent saves money. It solves a different problem: once you have already decided to field a team, you do not need every role burning top-tier tokens. Its precondition still holds: the lead must be strong enough, because a weak lead with cheap workers scores worse than a strong model working alone.
That sounds simple, but executing it is not. The lead first has to identify which stage is the bottleneck, the cheaper model actually has to be good enough at that stage, and the context handoff has to give workers neither too little information nor so much that they drown in it. Cohere used Luna for planning because planning on that benchmark is discovery work—parallelizable and forgiving of a lower model ceiling; move those cheaper models to implementation or verification, and they may not hold up. So this cannot be reduced to a one-liner like “throw research at a cheap model.” It is closer to a senior engineer managing junior engineers: first knowing what to delegate, then knowing how to scope the handoff, and finally knowing how to verify what comes back.
Even when you do field a multi-agent team, hold the line between parallel reading and planning on one side and single-threaded writing on the other: keep mutation authority with the lead, and limit sub-agents to gathering context, planning, and reviewing. If that boundary slips, coordination failures will quickly eat back whatever money you saved.
Put the empirical numbers side by side: Anthropic’s multi-agent system spends 15x the tokens of a standard chat turn, and both Vals AI and Cohere measured team costs at several times that of a single agent. Multi-agent has never been a free switch that makes a system better across the board; it is a trade that exchanges several times the token budget for specific engineering metrics. Seeing that trade clearly gives a team a stable rule for choosing architectures, instead of ripping and replacing the stack every time a new framework trends.
What that budget can buy is very specific: when subtasks are independent, it buys shorter end-to-end time; when information overflows a window or specs are sprawling, it buys wider search coverage and thorough verification; when multi-hour runs get stuck in a rut, forking and flushing history buys the diversity needed to escape an idea basin; when a context window is cluttered with trial-and-error residue, it buys a clean, isolated window for review. Each payoff has clear preconditions, and none of them triggers automatically on just any task. Capability routing is not on that list; it only changes how much a team costs after you have already decided you need one.
Conversely, if your task is bottlenecked on hard single-step reasoning, tightly coupled steps that cannot be cleanly split, or multiple roles trying to edit the same code at once, adding agents just repeats the same level of judgment several times over while footing the bill for step repetition, action-reasoning mismatches, and other coordination failures. Before adding a second agent to your workflow, identify where the bottleneck actually sits: unless you know which of those four metrics your 3x to 5x runtime cost is buying, start with a single agent and turn up the reasoning effort.