Over the past year or so of doing agent orchestration, I’ve always had a default assumption in mind: the model is a chip soldered onto the circuit board. How prompts are written, how retrieval recall is tuned, what tools are provided, how full the context window is packed—all optimization efforts occur outside the model. In an era dominated by closed-source APIs, this assumption matched engineering reality. Providers encapsulate models into black boxes, leaving developers with access only to peripheral plumbing.
Recently, while reading the AgentFlow paper collaborated on by several Stanford groups, a crack appeared in this solid iron plate. The paper’s most eye-catching headline is that a 7B model outperformed GPT-4o, but tucked away within the same orchestration system lies another, far more critical comparison. At the decision node responsible for planning and tool invocation, replacing a frozen Qwen2.5-7B with GPT-4o gained an average of only 5.8 points; allowing the 7B decision node to continuously learn from real tool feedback gained an average of 17.2 points. The gap did not stem from model scale; one was continuously absorbing feedback while the other remained frozen in place—that is the real watershed.
Open-weight models offer system architectures a new option: decision nodes within a workflow can be placed back into a real execution closed loop for training. This article reviews my process of unpacking this line of thinking, outlining the boundaries within which it works, and where it must stop.
Developers building agent systems have likely sensed this recently: the dividends of external orchestration are wearing thin. For the same task, using Claude Code today, switching to Codex tomorrow, and trying DSH the day after, one finds after cycling through them that casually tweaking prompts and tool descriptions often levels the differences. Continuing to stack context, add tools, and fine-tune retrieval yields an increasingly flat return curve. Every replaceable external tool has been swapped, yet the model inside each setup remains that unmovable constant.
This situation has its business logic. The delivery format of closed-source APIs is inherently static: developers pay per token to rent a black box, vendors complete training before shipment, and consumers can only intervene in how inputs are organized. Over time, model immutability shifted from a practical constraint into a default premise of system design. Few bother to ask anymore: must decision logic rely solely on hard-coded external rules?
The evolution of open weights is breaking this premise. Weights for models like Qwen, DeepSeek, and GLM can be freely downloaded and adjusted, and fine-tuning toolchains are no longer the exclusive preserve of a few research institutions. WebRL provides an earlier precedent: through self-evolving training, Llama-3.1-8B’s success rate on WebArena-Lite jumped from 4.8% to 42.4%, surpassing GPT-4o of the same period. Using training to enable smaller models to match active frontier models in specific scenarios has already been demonstrated. The current divide lies in determining which components to modify and whether the investment is cost-effective. The solution AgentFlow proposes is to focus training specifically on the decision nodes within the workflow.
The scope of modifications made by training is very narrow. AgentFlow comprises four functional modules: the Planner is responsible for decomposing subgoals and selecting tools—namely, the decision node mentioned earlier; the Executor handles tool execution, the Verifier assesses whether evidence is sufficient, and the Generator synthesizes the final answer. In the core experiments, all four modules uniformly adopt Qwen2.5-7B-Instruct, equipped with five tools: the default reasoning engine, Python code execution, Google Search, Wikipedia Search, and web retrieval. The four-module collaboration mechanism, internal memory format, available tool list, and inter-module transition logic were predetermined by the architect from start to finish, remaining untouched throughout training. Only a single localized part changed: the Planner’s internal policy for decomposing subgoals and invoking tools. The paper also leaves extending learning capabilities to other modules for future work; most components in the system did not participate in training.
From an engineering implementation perspective, the operational logic of this training mechanism is not complex. Given the same input problem, the system concurrently runs 8 real exploration trajectories, each actually triggering search engines, executing Python scripts, and verifying outputs. Upon task completion, the system assigns a score to the entire trajectory based on the success or failure of the final answer, and every decision step along the trajectory shares this outcome signal. The scores of the 8 trajectories are compared against one another: actions performing above average receive positive reinforcement, while those below average are suppressed. The paper names this mechanism Flow-GRPO, trading a coarse-grained signal shared across the entire trajectory for training convergence stability.
When judging whether training is truly effective, behavioral changes are far more reliable than aggregate scores. The MedQA medical QA benchmark provides clear micro-level evidence: before training, 66.2% of the decision node’s invocations were concentrated on general Google Search; after training, Google Search usage dropped to 10.9%, and the model autonomously shifted toward more specialized Wikipedia Search (rising from 0 to 59.8%) and targeted web retrieval (rising from 0 to 19.5%). No manual rules mandated this tool switch; real feedback reshaped its default behavioral inclinations. The model did not memorize more medical facts; what it learned was a preference to reach for domain-specific tools when encountering medical problems. Anyone working on orchestration has encountered this kind of stubbornness: with a tool plugged into the system, certain models refuse to invoke it no matter how the prompt is phrased, and multiple revisions of prompt engineering fail to override their default tendencies. With closed-source models, one simply has to endure it because the model cannot be touched; open weights paired with this training closed loop make this temperament and personality tunable for the first time. In the GAIA benchmark, tool invocation error rates dropped by up to 28.4%, showing that the model learned not only how to select tools, but also how to properly assemble invocation parameters.
What training modifies is the decision node’s selection policy; the topological structure of the workflow remains intact. The system’s skeleton—including module transitions, context management rules, and tool invocation specifications—still rests on the engineer’s upfront orchestration. Without these deterministic state tracking mechanisms, clear tool interfaces, and reliable verification standards, training would lack trustworthy success-or-failure signals. In my view, this conforms to the enduring division-of-labor principle in system design: facts that change dynamically and require continuous reference and correction remain outside the model, while stable, high-frequency, and objectively evaluable behaviors are absorbed into the weights. Invocation behaviors previously crammed into the harness via prompt rules and few-shot examples can settle into parameters once validated as stable, sparing the harness from repeatedly carrying this boilerplate logic with every request.
Under an identical system skeleton, the paper compares three ways of modifying the decision node, offering the most valuable comparative set in the entire work (Table 3). The baseline uses a frozen Qwen2.5-7B as the decision node, averaging 38.5 points across six benchmarks. The first modification upgrades the model: replacing the decision node with GPT-4o averages 44.3 points, adding 5.8 points. The second modification copies trajectories: collecting successful execution traces from GPT-4o and using supervised fine-tuning (SFT) to make the 7B imitate those steps causes the average score to plummet to 19.5 points, losing 19 points instead. The third modification is online training: letting the 7B decision node learn on the fly in a real execution environment reaches an average of 55.7 points, gaining 17.2 points.
These three attempts produced starkly different results. Swapping in a more powerful general-purpose model does not make it inherently suited for this specific workflow, with gains stalling at 5.8 points. Supervised fine-tuning that simply copies successful trajectories suffered a severe performance collapse. While counterintuitive at first glance, this outcome aligns with the nature of dynamic systems: every action an agent takes builds upon the new state it just triggered; when directly imitating someone else’s action sequences, the model never learns how to handle unexpected tool returns of its own. The SFT breakdown in the paper specifically pertains to distilling long-horizon planning decisions; on tasks with short interaction steps and clear outcome boundaries, SFT remains effective—a distinction that will be relevant later when discussing technology selection. What truly created the divergence was the third path: the exact same model, placed into the right feedback closed loop for training, gained 17.2 points.
The headline statement that 7B outperformed GPT-4o also needs calibration. It comes from the paper’s main table experiment: a tool-equipped four-module 7B system compared against a raw, tool-less GPT-4o outperformed it by 8.2 to 18 points across four domains. This system-level victory over a standalone model should not be overinterpreted. Other boundaries should also be clearly noted: the experimental comparison used mid-2024 GPT-4o, most benchmarks randomly sampled only 100 questions (AIME24 had only 30 questions), and all figures represent the authors’ paper self-reports, with no independent third-party replications to date.
Reading this far, many developers might consider introducing training to their own production agents. Before diving into engineering implementation, pass through four gates first; training becomes practically viable only if all four are cleared.
The first gate: recurring tasks. Training decision nodes requires upfront R&D investment and ongoing operational maintenance. If a task triggers only a few times a week, amortizing compute and engineering costs is difficult. Only high-frequency workloads with high call volume, stable business patterns, and long-term execution warrant considering training.
The second gate: automatically evaluable success and failure. Checking answers for math problems, running test suites for code, or validating database state changes in business systems—these scenarios naturally possess trustworthy scoring rubrics. Open-ended writing and research lack ground truth answers, where human evaluation is costly and hard to scale, and offloading judging to LLMs easily introduces scoring bias. The search, math, and scientific QA tasks chosen by AgentFlow all fall into domains where automatic evaluation is straightforward; this choice of tasks is a prerequisite for the method’s viability.
The third gate: error bottlenecks concentrated in decision nodes. What training can improve is the logic for decomposing goals, selecting tools, and determining when to terminate. If reviewing failure logs reveals that the crux lies in insufficient retrieval recall, frequent third-party API errors, or context window management failures, training a decision model will not solve these problems; what should be optimized is the external engineering infrastructure.
The fourth gate: environments supporting safe resets. Online training relies on tens of thousands of real invocation interactions, inevitably accompanied by numerous failed attempts. Whether invoked tools cause irreversible side effects and who bears the cost of failures are essential considerations. Without an isolated, repeatable, and quickly restorable sandbox environment, training overhead can easily spiral out of control, and the model might even learn to exploit loopholes in the scoring rules.
After clearing the four gates, there is still a minimal validation step that requires no GPUs: extract a batch of real failure cases from the existing agent’s historical logs, manually summarize failure causes (distinguishing between missing knowledge, tool errors, formatting issues, and decision missteps), simultaneously confirm whether success or failure can be evaluated automatically, and finally run a round each with a smaller model and a larger model under identical tools and invocation budgets to establish baselines. Once attribution and baselines are complete, whether to proceed with training usually becomes immediately obvious.
The native AgentFlow setup is a typical lab-grade configuration: 8 A100 GPUs, 8 concurrent exploration trajectories per sample, synchronous waiting for tool execution, and real-time judging by GPT-4o (training configuration). Typical engineering teams do not need to replicate this heavy architecture; today’s technology stack offers entry paths with much lower barriers.
Path one: overlaying a training interface onto existing agent code. Unsloth paired with OpenPipe’s ART is currently a low-friction combination (official claim). Business code requires no refactoring—simply wire up trajectory collection and scoring functions—while the training backend can switch between a local 24GB VRAM GPU, a rented single A100, or W&B Serverless RL (free training during preview, formal pricing TBA). OpenPipe’s founder shared a case study: a 14B parameter email retrieval agent achieved 96% accuracy after training on a single A100 (semi-independent practice). Community developers have also used an RTX 4090 to teach an 8B open-source model basic tool invocation in one hour (independent practice). These empirical data points point in the same direction: the entry threshold is lower than expected.
Path two: adopting a decoupled training framework like Agent Lightning (official claim). It shares a direct ecosystem relationship with AgentFlow, with AgentFlow itself officially indexed as one of its community projects. The core concept is decoupling the training engine from the agent runtime, enabling zero-code-change integration with existing agents. In an official case study, a 9B model using 6,000 training samples increased the SWE-bench Verified solve rate from 41.8% to 56.4% (paper, official claim). It also supports Thinking Machines’ Tinker as a training backend via a pay-as-you-go API model, completely eliminating the need for local GPUs, though specific pricing should be verified independently.
Path three: starting with two-stage lightweight fine-tuning. First run supervised fine-tuning overnight using LoRA on consumer-grade GPUs (gpu rental costs around $5 to $30), aiming to internalize input and output formats for tool invocation without burdening the model with long-horizon planning too early. Solid evidence for this comes from SWE-Gym: 491 rigorously filtered interaction trajectories provided an absolute gain of 12 to 14 percentage points for Qwen2.5-Coder-32B on SWE-bench Verified (ICML 2025 peer review). Combined with the boundaries outlined in the earlier comparative experiments: SFT is effective at cementing short-horizon reliability behaviors, whereas long-horizon decision distillation is prone to failure. Treat this as the first-stage leverage; once formatting and invocation stability meet requirements, upgrade to online training based on business needs.
The shared prerequisite for all these paths remains the four gates. Before meeting the threshold, first refine retrieval quality and tool ecosystems; after clearing the threshold, begin experimenting with the localized business modules featuring the most concrete scoring criteria.
On the performance front, even after discounting bonuses from external tools and nuances in evaluation protocols, the net gain of 17.2 points brought by online training under an equivalent system architecture still firmly holds. On the cost front, the ledger remains far from settled. The paper only discloses 8 A100 GPUs, without revealing total training duration, idle time spent synchronously waiting for tool executions, accumulated billing for GPT-4o judge invocations, or how much per-task cost in production is saved compared to directly calling commercial APIs. In recent years, commercial API prices have continued to decline, while inference costs for hosting open-source models have also dropped. The token cost savings from an in-house training setup must first offset the full lifecycle expenses of dataset construction, environment setup, iterative training and evaluation, and ongoing maintenance before the economic calculation becomes convincing. High-volume, stable business workflows might make the math work; low-frequency, variable long-tail scenarios are almost certainly uneconomical.
The core incremental value of open weights lies in restoring a previously locked-down policy optimization layer to engineering systems, rather than merely offering a cheaper deployment option. External orchestration governs the deterministic skeleton, encompassing state transitions, tool permissions, memory storage, and endpoint verification; training takes over the decision nodes that are difficult to exhaustively cover with hard-coded rules. When planning your own agent systems, if a core node is called frequently, evaluated objectively, and bottlenecked by decision logic, select it as your first experimental target. Before that, solidify state tracking and automated evaluation before talking about training.