Self-Improving AI: A Flattened 2D Field

First Reaction: Wow, That’s Impressive

In the first half of 2026, self-improving AI transitioned from a laboratory term into the headline of fundraising news. In May, Recursive Superintelligence (RSI) came out of stealth: $650M raised at a $4.65B valuation, with 8 co-founders including Richard Socher and Yuandong Tian. That same week, Karpathy’s open-source autoresearch enabled an agent to run 100 experiments overnight. In July, OpenAI demonstrated GPT-5.6 Sol autonomously completing the post-training of another model, and RSI signed another $410M AWS compute deal. In a TechCrunch interview, Socher discussed the company’s expansion: “less about headcount and more about agent count”.

With these news stories popping up one after another, the initial reaction is inevitably: AI has truly started modifying itself, and this is probably a holy grail that only frontier labs can touch. This excitement is grounded in facts, but if we simply follow our intuition, it is easy to compress several distinct engineering practices into a single term—unwittingly elevating the barrier while obscuring the true landscape of the field.

Sitting down to examine the systems available on the market one by one reveals several things the headlines didn’t tell us. This field is more real, more accessible, and more crowded than it appears on the surface. From Weco’s $8M seed round to RSI’s $650M, all the way to internal projects at frontier labs, the explorations here span a wide spectrum; small teams are already delivering verifiable real-world value, such as Weco’s system beating a baseline manually tuned by human engineers for 2 years. Progress here is also quite solid: not only is it setting new SOTAs on benchmarks, but over 80% of the code merged into Anthropic’s official codebase since May 2026 has been written by Claude. Venture firm NEA has even listed self-improving systems as an independent sub-track, profiling 7 representative companies.

Precisely because this field is both real and crowded, with engineering approaches diverging significantly across players, a positioning lens is needed to see the full picture clearly. Breaking down seemingly complex systems into two dimensions—what is being modified and what signal is used for verification—helps us move beyond scattered news impressions and build a systematic understanding of the entire landscape. Through this coordinate frame, we can see the technical convictions of different companies, understand where they believe the bottlenecks lie, and see what directions they are betting on. Equipped with this lens, whenever a new technical claim or startup emerges, we can place it into its corresponding cell, understand the distribution of the entire field, and see through the underlying logic.

Before mapping all the players onto a panoramic matrix, it is worth starting with three of the most representative systems. Although they all share the halo of self-improvement, in terms of concrete engineering implementation, they are actually doing three entirely different things.

The Same Term, Three Completely Different Things

Running several systems that all wear the self-improvement label side by side reveals that what they do varies drastically. The most lightweight is autoresearch, open-sourced by Andrej Karpathy. Its design is simple: the agent only touches a single code file called train.py, which contains the GPT model’s architecture definition, optimizer configuration, and training loop. Every time the agent modifies the code, it runs a 5-minute miniature training run and checks the compression loss metric val_bpb on the validation set (bits per byte; lower values indicate better model compression and prediction quality). If the metric drops, the change is kept; if it increases, it is immediately rolled back, and the agent moves on to the next attempt. The system runs about 12 experiments per hour, accumulating around 100 iterations overnight. Throughout the entire process, humans are only responsible for writing guiding rules in a program.md document. In Karpathy’s tests running continuously for about 2 days on a depth=12 nanochat model, autoresearch automatically discovered around 20 code modifications that reduced validation loss, and all of these changes successfully transferred to a larger depth=24 model.

One level up is AIDE² by Weco. It employs a two-level loop architecture: the inner loop agent debugs and optimizes specific business code against existing eval sets, while the outer loop specifically circles back to modify and optimize the harness scaffolding used by the inner agent. In an 8-day, fully unattended test without human supervision, the outer loop executed 100 consecutive steps, and the system automatically discarded roughly 90% of proposals that failed validation. After these 8 days of autonomous iteration, the resulting AIDE85 version surpassed the AIDEhuman baseline—which had been manually tuned by a team of human engineers over 2 years—across three standard benchmarks, including MLE-Bench Lite.

At RSI, which raised substantial funding, the scope of modification shifts again. While many intuitively assume the model is modifying its own weights, RSI modifies training scripts, model architecture source code, and underlying GPU kernels. Across the 3 benchmarks published in its first official technical article, the system achieved a 1.3x training speedup on the NanoChat task, compressed NanoGPT training time from 79.7 seconds to 77.5 seconds, and narrowed the gap between performance and the theoretical physical hardware limit by 18% on the SOL-ExecBench benchmark. The entire system operates as a standard scientific research closed loop: autonomously generating hypotheses, writing code, deploying and running experiments, collecting results, and using feedback to plan the next experiment. In the NanoGPT optimization case, the final solution spanned both high-level training scripts and low-level kernel files within approximately 200 lines of code changes.

Placing these three systems side by side makes the differences clear: Karpathy modifies single-file code, driven by a single scalar metric from 5-minute miniature training runs, relying on human-written top-level rules; Weco pivots to the agent’s execution harness, relying on dual public/private cross-evaluations to achieve 8 days of autonomous screen-free operation; RSI achieves system-level search spanning training code and operator kernels, relegating humans to the boundary layer of setting objectives and reviewing outputs.

To make sense of these diverse systems, the most direct starting point is to clearly understand: during the iterative process, what exactly are they modifying?

Dimension 1: What Is Being Modified

When talking about self-improvement, intuition immediately brings to mind models directly modifying their own weights. In real-world engineering exploration, however, the targets of modification are distributed across several clear tiers.

The most direct level is indeed weights. Yet even at the weight level, current exploration exhibits characteristics of division of labor and collaboration, rather than a model modifying its own parameters. In July 2026, OpenAI showcased such a process in a technical demonstration: GPT-5.6 Sol received an ambiguous instruction via Codex described as a “fairly under-specified prompt,” and subsequently took over the entire post-training workflow of another smaller model, Luna, fully autonomously. Sol independently selected training configurations, allocated GPU compute, launched training scripts, and monitored metrics without human intervention. OpenAI researcher Kathy Shi remarked that such tedious post-training work previously was “the work of a team of senior researchers, and now automated researchers are getting very close.” In this case, Sol improved the separate Luna model without modifying Sol’s own weights.

Following this logic further, more systems avoid touching runtime network parameters altogether, choosing instead to directly evolve the source code that carries the algorithmic and engineering logic. The aforementioned Karpathy autoresearch is a prime example of single-file code evolution, exploring better architectures through repeated edits to train.py. In industry-grade frontier exploration, DeepMind’s AlphaEvolve follows the same path. AlphaEvolve is a Gemini-powered code evolution agent equipped with an automated evaluator to verify the correctness of solutions. In real-world production, the compute kernels optimized by AlphaEvolve sped up Gemini training by 23% (one-year retrospective). I previously wrote a standalone piece on its mechanism: the LLM proposes candidates while the evolutionary framework decides survival, and the true breakthrough lies in separating the two.

Beyond algorithm code itself, a number of teams have recognized that the performance of large models heavily depends on their peripheral engineering environment, thus shifting their focus of modification to toolchains and execution environments. Japanese startup Sakana (which established a dedicated RSI Lab) developed DGM (short for Darwin Gödel Machine), showcasing this tool-level evolution. While keeping LLM weights frozen, DGM allows agents to autonomously modify their own Python codebase, including rewriting tool interfaces, optimizing workflows, and redesigning validation checkpoints. In testing, DGM raised the SWE-bench resolve rate from a baseline of 20.0% to 50.0%. Sakana’s commercialization direction is another product, Fugu, which trains multi-agent orchestration directly into model weights, which I also deconstructed previously. Commercial startup Poetiq has also embraced this route; the company explicitly states that none of its published SOTA results involved adjusting underlying LLM parameters, with its Metasystem focusing instead on optimizing the harness environment and internal workflow logic for various benchmark tasks.

Inside leading frontier labs, the automated generation of scaffolding and engineering code is also penetrating rapidly. Official data released by Anthropic shows that since May 2026, over 80% of code merged into its official codebase has been written entirely by Claude. In a constrained internal experiment at Anthropic, researchers had Claude attempt to optimize training code for a small model: Opus 4, released in May 2025, achieved approximately 3x speedup on this task, while by April 2026, Mythos Preview reached roughly 52x speedup on the same code. Anthropic noted in its report, however, that this was a miniature experiment and cannot be simply extrapolated into generalized speedup capabilities.

Yet among all these targets of modification, the one most easily overlooked in public discourse—and the one that constitutes the critical bottleneck in practical system building—is the evaluator. A large survey report covering 1,250 arXiv papers categorizes self-evaluation as a distinct core category, including 318 dedicated papers, 82% of which were published in 2026. After analyzing numerous failure cases, the authors noted: “one bottleneck recurs everywhere: the evaluator.” In 2026 alone, at least 5 independent architectures emerged in the industry dedicated to exploring co-evolution between evaluators and the evaluated models. Modifying the evaluator, while lacking headline-grabbing hype, is a standalone track that determines whether an entire self-improvement system can hold up.

Dimension 2: What Signal Drives the Modification

When discussing self-improvement, many people’s initial reaction is to categorize by data source: whether it uses massive code data, model-generated synthetic data, or expert-annotated data. In its marketing, commercial team Poetiq highlights real-world business data as a core selling point, emphasizing to enterprise clients that providing actual business problems and a few hundred real examples is sufficient to bootstrap optimization—without requiring the thousands to millions of labeled samples that traditional fine-tuning or reinforcement learning rely on.

Data sources, however, do not constitute an effective classification dimension. Two systems using the exact same code data can differ by an order of magnitude in improvement capability if their verification mechanisms differ; even when fed high-standard expert-annotated data, a system can easily spin in circles on complex tasks without clear verification signals. What truly determines whether a system can continuously evolve is the strength of the verification signal obtained after making modifications—that is, what it relies on to confirm that a change is genuinely effective.

In that academic survey covering 1,250 papers, the researchers organized verification signals from strongest to weakest into five tiers:

The most solid tier is the formal verifier, which provides mathematical proof-level deterministic true/false feedback with zero ambiguity; Down from there is execution feedback provided by computational environments, including whether code compiles and runs, passes unit tests, and shows objective score changes on benchmarks; Further down are reward models, dedicated scoring networks trained on preference-annotated datasets that provide continuous quality estimates for open-ended outputs; In the absence of a dedicated reward model, it falls back to an independent general-purpose large model acting as an LLM judge for subjective review and scoring; The weakest tier is self-evaluation, where the very same model making modifications evaluates its own generated results.

Empirical analysis from the survey shows that all experimentally verified self-improvement gains strictly decrease along this verification signal gradient. On the weakest end of self-evaluation, the SkillsBench test set provides a clear comparison: using skills written by human experts increased the model’s task pass rate by 16.2 points, whereas skills generated by the LLM itself brought no measurable performance gain in practice. When a system acts as both player and referee, verification signals easily distort. A more complete public case is Dan Austin’s ai-trains-ai, which connected an outer training agent, inner post-training, and hidden evaluations into a feedback loop; however, when the evaluator mismeasured, the reward continued to climb anyway (I previously deconstructed this piece).

When classifying these 1,250 papers, the authors also distilled a highly isomorphic two-axis framework: the first dimension focuses on the target of modification, categorized into 393 papers on test-time / deployment-time self-evolution, 340 on training-time self-iteration, 318 on self-evaluation, and 139 on automated research; the second dimension evaluates reliance on human intervention, divided into human-in-the-loop (every modification requires human confirmation), human-on-the-loop (humans only set boundary objectives and review outputs), and fully autonomous closed-loop systems.

The taxonomy distilled by academia from this vast sample of literature aligns remarkably well with actual industry trajectories. In this panoramic coordinate system, the degree of human closed-loop intervention can serve as an in-cell attribute indicating whether a system in practice relies on in-the-loop humans, on-the-loop oversight, or operates entirely without human involvement.

Combining the targets of modification and verification signals reveals distinct distribution characteristics in player and project density across the field: in cells such as code writing, operator optimization, and standardized benchmarks where strong execution feedback is available, explorers are densely clustered—the autoresearch lineage, DGM lineage, Poetiq, and Weco are all crowded here; in high-difficulty cells like open-ended scientific discovery where deterministic verification signals are lacking, exploration is extremely sparse, and fully closed-loop automated research projects appear exceptionally thin.

The Matrix: Where Everyone Stands

Putting these two dimensions together allows us to map the industry’s representative key players onto a single panoramic matrix. From minimal open-source experiments to startups with hundreds of millions in funding, to internal tools at top frontier labs, what each is modifying, what signals they verify with, and how far humans have stepped back become immediately clear:

From a capital and market structure perspective, venture firm NEA has already categorized self-improving systems as an emerging startup sector in industry reports, profiling 7 representative startups. The funding spread across these teams is pronounced, ranging from Weco’s $8M seed and Poetiq’s $45.8M seed all the way to RSI’s $650M round ($4.65B valuation). It is worth noting that there is also Ricursive Intelligence, a software company focused on electronic design automation (EDA) that raised $335M ($4B valuation); despite having a similar name and abbreviation, it is an entirely separate commercial entity.

The distribution of player density across the matrix clearly reflects the constraints imposed by verifiability. The execution feedback and standardized benchmark columns gather the vast majority of active industry players; the formal verifier column has very few real-world cases due to extremely high technical barriers and narrow application boundaries; and at the tail end that relies entirely on self-evaluation, industry has generated virtually no serious commercial implementations.

More crucially, the industry currently faces a pervasive Level 1 ceiling. The Weco team outlined a clear four-level evolutionary ladder in their technical roadmap: Level 0 is delegation: the system acts as an auxiliary tool, with key decisions finalized by humans; Level 1 is net positive: modifications yield definite performance improvements, but the agent performing the modifications does not itself become more capable; Level 2 is ignition: the capabilities of the agent responsible for improvements increase with each iteration, driving the system into a flywheel state of positive self-acceleration; Level 3 is inflection.

Measured against this standard, all current players and systems remain stalled at Level 1. Even Weco, which first introduced this progression, hit a wall in internal tests attempting to reach Level 2: when using the evolved AIDE47 model as the outer loop to guide self-improvement, experiments showed that this setup merely accelerated search and generation speed without showing statistically significant advantages in the final quality of generated solutions. Weco’s CEO reflected that the compute and engineering costs required to construct infinitely nested meta-loops expand exponentially in practice.

In academic frontier exploration, the representative system in published literature that comes closest to a fully unattended, end-to-end closed loop is A-Evolve-Training. The system attempted to autonomously execute the complete post-training workflow for a 30B-parameter model, operating entirely without human intervention across 4 iterations over several weeks, ultimately achieving an evaluation score of 0.86—close to the 0.87 score achieved by top human experts through manual tuning. The system even demonstrated the ability to self-detect when proxy metrics decoupled from true performance and actively self-correct. However, as the sample pushed furthest to the right frontier among the 1,250 papers, it still cannot function without calibration against external preset benchmarks, falling short of a closed-loop system entirely free from external intervention.

Empirical tests from skeptics point to the same signal. Princeton University conducted a shadow evaluation experiment: the research team had Claude Opus 4.8 conduct open-ended research over 6 days with a budget of $3,000 plus GPUs; the resulting two papers were rejected outright by the original authors according to conference standards.

Prominent AI industry observer Jack Clark interpreted Princeton’s evaluation results as a significant cooling signal on the short-term timeline for recursive self-improvement. CSET Director Helen Toner also questioned current claims: “I think that is different from the classic definition of RSI, which is really that there are no humans needed.” In August 2026, MIT Tech Review published an in-depth review as well, with a headline pointing directly to “AI’s recursive self-improvement might not come so quickly after all”.

Placing this cooling evidence back onto the coordinate matrix reveals that all setback cases and counter-evidence are clustered in the open-ended research cells that lack objective verification signals. The high-density clustering in strong verification signal cells and the roadblocks in open-ended research cells reinforce each other mechanistically.

How to Use This Map

Faced with endless claims of self-improvement breakthroughs and fundraising news, I later found that no matter how grandiose the PR press release or how high the reported valuation, simply locating a new system along three specific questions quickly places it onto the map:

First, what is the object being modified by the system? Is it neural network weight parameters, concrete training and architecture code, the harness scaffolding the agent runs on, or the evaluator sitting at the bottleneck? Second, what strength of signal does the system rely on to verify the validity of improvements? Where does it land across the five-tier gradient of formal verifiers, execution feedback, reward models, LLM judges, and self-evaluation? Third, where do human engineers stand in the system’s operation? Are they in-the-loop confirming every step, on-the-loop managing only boundaries, or is it completely human-free as claimed?

Clarifying these three questions brings the true coordinates on the map of any claimed self-improving technical solution into clear focus. The interpretation heuristic is straightforward: whenever encountering a claim of achieving recursive self-improvement, first probe the strength of its underlying verification signal. If the verification signal is built on formal verifiers or strict code execution feedback, the claim rests on a solid engineering foundation and warrants closer scrutiny against benchmark data; if the verification signal relies primarily on the model’s own self-evaluation or fuzzy subjective scoring, such claims are often largely commercial packaging.

At the edges of this map lies a core blank space. In the systematic survey covering 1,250 papers, scholars listed the exchange rate of grounding as the number one open scientific question: theoretical analysis indicates that purely closed-loop systems without external interaction inevitably suffer model degradation, which is why teams in practical engineering universally inject external ground-truth signals ad hoc to maintain the system’s evolutionary direction; however, the minimum proportion of external signal input rate required to prevent capability decay remains quantitatively unmeasured across the industry. Whether the Level 2 ignition stage requires continuous injection of such external signals, and how much external signal is needed to support it, remains the emptiest and most crucial unknown on the entire map, awaiting further observation.