Not long ago, building a coding agent took little more than a large language model and a 100-line main loop. Today, if you look at open-source engineering frameworks from major labs or popular community repositories with tens of thousands of stars, the wrapper layer around the model responsible for tool execution and context management—known in the industry as the harness—has grown massive, packing in dozens of skills, multi-subagent committee mechanisms with a dozen or more subagents, and all sorts of context-compaction hooks. True, if you run an agent for hours on a large task using only a bare-bones main loop, it will easily stall or drift off course; yet when individual developers want to build a practical harness of their own, facing all this elaborate engineering makes it hard to tell which parts genuinely boost scores and cut costs, and which are just ornamental bloat that inflates the bill.
In early November 2026, data evaluation company Surge AI and the FireFlash project from a joint UCLA team released empirical reports in the same week. Both teams took apart popular workflow plugins and low-level harness techniques one by one, running controlled experiments on real-world engineering problems and long-horizon terminal tasks to filter out the engineering tricks that actually work.
Getting today’s flagship models to write a localized snippet of code is easy; the hard part is modifying a large codebase without missing upstream and downstream hazards. Surge AI worked with senior engineers to extract 60 high-level engineering tasks from real companies’ production codebases, creating an engineering benchmark called sudo L7. Each task is graded out of 100 points with a passing bar of 75, using rubrics as unforgiving as a senior engineer’s code review: if you only check whether the immediate error is fixed and the local feature runs, several leading models score around 80%; however, the rubric also penalizes models for breaking downstream callers, reinventing the wheel instead of using existing utility functions in the codebase, or failing to spot hidden business-logic flaws in the prompt itself. Once all those engineering deductions are factored in, the share of tasks crossing the 75-point passing bar plunges to roughly 30% to 40%.
To tackle this problem, the most popular approach is to bolt on a multi-agent plugin that simulates meetings at a software company. Take the popular open-source Compound Engineering plugin, which provides 37 skills and orchestrates multiple agents sequentially through planning, execution, and review stages. In Surge AI’s harness benchmark experiments, one task involved fixing a bug in an event registration system where, after a user filled out registration details, the system sent out a confirmation email containing the online meeting password before payment was completed.
When handed this task, Compound Engineering dispatched 17 subagents to review the plan, simplify the code, and check code quality. Unfortunately, every single subagent signed off on the review, and not one noticed that the confirmation email was still being sent before payment. Because the primary agent that drafted the initial plan overlooked payment timing, all the subagents merely reviewed code formatting against that same flawed plan.
Across the full benchmark, this elaborate plugin drove per-task API costs up to 8x the baseline, while nudging the score up only slightly from the default baseline of 31.1 to 32.9—a difference well within random noise. Even the plugin authors recently merged an update restricting when subagents are spawned, reserving subagent dispatch for cases where it actually pays off.
Rather than spending heavily on a dozen shallow-thinking clones to
hold meetings, it is far more effective to invest that budget directly
in a single model’s reasoning depth. Often the most direct move requires
no external plugin at all—just turning the same Claude Opus 5.5 model’s
reasoning effort up from the default medium tier to the high tier
xhigh. On another task involving a video editor that
crashed when the first frame was missing, the medium-effort model simply
patched the immediate error and submitted early, scoring just 14 points.
Switched to xhigh reasoning, after fixing the first error,
the model traced the fallback path one step further downstream,
discovered another code path that also assumed the first frame was
always present, fixed both hazards together, and earned a perfect 100
points—for just $0.29 more on that task.
Aggregated across the entire benchmark, installing no flashy plugins and simply dialing up the single model’s reasoning effort jumped the overall score to 42.4. That 11.3-point leap cost only 3.5x the baseline, leaving the multi-agent committee approach behind in both verification thoroughness and cost-effectiveness.
Once a single model’s reasoning effort is dialed up, its single-step reasoning is already plenty deep, yet two classic failure modes still crop up across multi-turn loops: jumping straight into editing code without writing tests when handed an ambiguous prompt, and declaring victory too early right after fixing the first error.
Fixing these two problems does not require a multi-agent committee either—just two small adjustments inside a single main loop. The open-source plugin Superpowers strips away complex multi-agent review panels and enforces one core rule: clarify the specification before writing feature code, and follow a strict test-driven development loop of writing a failing test first and then writing the code to make it pass. That change added only 10% to API costs while lifting the overall score by 2.4 points (to 33.5), achieving a statistically significant gain.
The second effective checkpoint prevents premature submission. Built
into Claude Code, the /goal
objective-verification command, paired with a code-review plugin,
runs an extra completion-criteria check whenever the model tries to stop
and submit at the end of a turn. If it detects that the original goal
has not actually been met, it bounces the task back into the next turn
so the model keeps working. This simple end-of-turn feedback loop used
only 1.8x the baseline cost while boosting the overall score by 3.3
points (to 34.4), earning the highest score among all external workflow
plugins.
The lighter the control layer, the better—a finding echoed in UC Berkeley’s HarnessTax evaluation report. Comparing heavyweight commercial harnesses against a minimal 100-line harness across 492 standard coding problems, the researchers found that heavyweight harnesses consumed nearly 20x more tokens in first-turn system prompts and tool definitions, making per-task bills 3-5x more expensive while yielding an accuracy difference of less than 2%.
If the control flow is that simple, why is the low-level code inside production-grade harnesses still so thick? Because on long-horizon terminal tasks, even when a model is locked to its highest reasoning effort from the start, the physical constraints of the terminal environment can still cause it to stall mid-run. In the FireFlash project’s long-horizon agent technical report, the research team used the open-source GLM-5.3-Flash model locked at maximum reasoning effort and gave it only 4 basic tools: running terminal commands and reading, writing, and editing files. Yet when run inside the default official Claude Code harness on Terminal-Bench 4.0, it scored only 35.8; swapping in the team’s modified low-level harness—without any model training—pushed the score to 40.2.
Those extra 4.4 points came entirely from low-level harness plumbing that shielded the model from three specific failure modes in long-horizon tasks. The first failure mode is terminal log flooding and premature timeout kills: a large build or test suite can dump tens of thousands of lines to stdout, flooding the context window if injected directly into the conversation; and if a slow build hits a timeout threshold, a standard harness kills the process outright, wiping out all prior progress. FireFlash instead redirects every command’s stdout to a temporary file by default, showing the model only a short truncated preview alongside the file path so it can read details on demand; when a slow command hits the timeout threshold, rather than killing the process, the harness automatically moves it into the background to continue running asynchronously.
The second failure mode is context compaction: once a long task approaches 85% of the context window, the conversation history must be compressed. Deleting older turns and replacing them with a summary rewrites the beginning of the conversation, shattering the prefix cache that large models rely on to save compute and forcing every subsequent turn to be recomputed at full price. FireFlash uses a clever alternative: it keeps the most recent raw turns intact, compresses older history into a summary, and appends that summary via an append-only segmented prompt structure so that context space is freed while preserving the prefix cache. Those three plumbing upgrades alone—backgrounding slow commands, writing long outputs to files with previews, and append-only summarization—cut the cost per rollout by roughly 1/3.
With the plumbing cleared, the team targeted the third failure mode
by adding three anti-waste rules to the prompt. Agents in a terminal
routinely burn money on three foolish habits: polling background jobs in
tight sleep loops, rewriting an entire large file from
scratch just to change a few lines of code, and writing to the same file
in fragmented chunks across multiple turns. Once the prompt explicitly
banned sleep polling, mandated in-place editing tools, and
required single-pass file writes, single-run costs dropped by an
additional 16% on top of the 1/3 reduction.
After fixing the low-level plumbing, the FireFlash team used reinforcement learning to add another 6.0 points in the second half of the climb, pushing the terminal benchmark score to 46.2. At just $0.72 per run, their open-source recipe overtook the proprietary Fable 5 system (44.5 points at $22.02 per run).
That second-half gain hinged on an easily overlooked detail: out of the box from pretraining, the model had never seen this custom append-only summary format or truncated file-preview layout. To help the model truly internalize the custom plumbing, the team wrote an automated unit test suite that strictly locked the harness used during reinforcement learning rollout sampling to match the harness used during final benchmark evaluation, guaranteeing that code logic and prompts on both sides were identical down to the word. They then ran full-parameter reinforcement learning on 64 GPUs for under 5 days while awarding a length bonus to rollouts that solved tasks with fewer tokens, compressing token consumption on successful trajectories to 84% of its prior level and teaching the model to work fluidly with the underlying plumbing.
This also explains why open-source harnesses from frontier labs often look so arcane today: top teams now co-design their low-level plumbing and model post-training as a single tightly coupled system. Yet for individual developers who have no need to train models from scratch, once you strip away that complex outer shell, the techniques that actually survive benchmark scrutiny are already within reach. Instead of assembling a bureaucratic committee of a dozen subagents taking turns in meetings, you can capture the core gains of an industrial-grade harness in just a few hundred lines of clean code: turn up a single model’s reasoning effort first, guard the main loop with two lightweight checkpoints—writing failing tests first and verifying the goal at the end of each turn—and bake file-backed output previews, automatic backgrounding for slow commands, cache-preserving append-only summarization, and anti-polling rules right into your foundation.