Inference & PerformanceAI CodingIndustry & Competition

Pitching a Three-Day Nvidia Replacement While the Slide Footnote Says Prefill Only: The Real Boundary of AI-Written Kernels

On November 10 this year, Anthropic officially kicked off its IPO roadshow and posted a public roadshow video on its website. In the presentation, co-founder and Chief Compute Officer Tom Brown noted that bringing up a new chip typically takes dozens of engineers several months. Using their internal model Mythos, however, the team got Claude running on AMD GPUs in just 3 days with virtually no human intervention, with the model independently writing nearly 100% of the low-level kernels.

Low-level kernels are small compute routines that run directly on GPU hardware to handle foundational operations like matrix multiplication. After the clip circulated, posts with over 500,000 views flooded social media proclaiming that Nvidia’s CUDA moat had vanished. Yet a check of the full roadshow transcript shows that neither Nvidia nor CUDA was mentioned once.

Pausing the video to read the gray fine print in the bottom-left corner, or opening the amended S-1 registration statement filed with the SEC on the same day, reveals a completely different story. In that regulatory filing, the terms “3 days” and “kernels” do not appear a single time. Meanwhile, the concurrent private placement roster disclosed on the same day shows Nvidia and Google tied for first place, each committing $10 billion to purchase company stock. Why did the roadshow play up the feat while the statutory filing stayed silent and Nvidia still wrote a massive check? The answer sits in two lines of fine print at the bottom left of the slide.

What the Two Lines of Gray Fine Print Quietly Carve Out

Returning to the slide captured in the annotated roadshow transcript, the orange curve on the left peaks at roughly 3 days with Mythos, while the green curve on the right marks 6 to 12 months for traditional methods. However, the gray footnote in the bottom-left corner attaches three qualifiers: the result applies to a single model, runs on a single multi-GPU AMD MI355X server, and brings only prompt processing to usable performance.

The most critical restriction here is that it covers only the prefill stage. As explained in our earlier piece on prefill-decode disaggregation in large language models, processing a conversation in a large model splits into two distinct phases. The first phase is prefill, referred to as prompt processing in the slide footnote: the model reads the entire user prompt at once, packing the input text into a large matrix and computing it in a single pass, which primarily taxes single-GPU compute. The second phase is decode, where the model writes out its response one token at a time; generating each new token requires reloading hundreds of gigabytes of model weights and key-value cache from GPU memory, making memory bandwidth and cross-chip communication the primary bottlenecks.

Why could an AI write the low-level kernels for the prefill stage in 3 days? Single-node prefill is a self-contained practice range: all prompt computation happens on one machine, each kernel spans just a few hundred lines of code, and inputs and outputs are cleanly structured. Engineers can treat the output from Nvidia GPUs as the ground-truth answer key, let the model iterate on code and run tests around the clock, and get compiler feedback in seconds. That closed loop with a clear objective and rapid automated grading sits squarely inside a coding agent’s comfort zone. By contrast, the 6-to-12-month traditional curve on the right covers the full production timeline required to bring up prefill, decode, and large-scale training end to end. Comparing 3 days of single-node prefill bring-up against half a year of full-stack production deployment is like comparing a warm-up sprint to an entire championship season.

Engineering boundary comparison between a closed single-node prefill sandbox and full-stack production deployment

Beyond Single-Node Prefill: What the Real Multi-Chip Production Traps Look Like

If low-level kernels for the prefill stage can be brought up in a few days, what are human engineers wrestling with for the remaining half year? Once a system moves from single-node prompt reading to multi-chip decode and large-scale training across dozens of accelerators, the nature of the problem changes completely. A frontier model cannot fit on a single GPU and must be sharded across dozens of chips computing in parallel. For every new token generated, those dozens of chips must synchronize data at high speed while aligning probabilities down to several decimal places to select the highest-scoring token.

Anthropic’s postmortem of three recent production issues, published this September, illustrates what this deep end actually looks like. At the time, many developers complained that Claude had suddenly degraded or started outputting garbled text. After weeks of investigation, the team found that not a single bug came from single-node prefill kernels; every one of them stemmed from multi-chip decode sampling and underlying compiler behavior. Two of those incidents occurred directly on Google TPU servers running in multi-chip clusters.

The first failure originated in the decode stage on Google TPU servers. After the engineering team deployed a runtime optimization, the model occasionally assigned high probabilities to rare tokens while generating English code. Users writing code in English would suddenly see a Thai greeting or a Chinese character appear mid-output, and it took a full 8 days to roll back and fix the issue.

The second bug was even more elusive and lay dormant for a full 8 months. During decode across a TPU cluster, dozens of chips each computed token probabilities in 16-bit floating point before aggregating and sorting the results, but the underlying TPU compiler enabled excess-precision optimizations by default and automatically promoted part of the computation to 32-bit floating point. Because decimal precision no longer matched across chips, the devices disagreed on which token ranked first during aggregation and dropped the highest-probability token outright.

When engineers rewrote the sampling code to patch that precision mismatch, they triggered another hidden compiler bug inside an underlying approximate top-k sorting algorithm. The code behaved normally on standard processors and failed on TPUs only at specific request batch sizes. Enabling debugging tools or altering the order of computation steps caused the bug to vanish immediately. The team ultimately had to abandon the fast approximate sort and fall back to exact sorting. Tracking down and fixing cross-chip bugs that trigger only at specific batch sizes and disappear the moment a debugger attaches cannot be solved by a few days of single-node code generation.

After the Prospectus Dropped the Three-Day Pitch, Who Built the Testbed Backing Up the AI?

Turning back to the amended prospectus, which is governed by strict securities disclosure rules, Anthropic omitted the short-term porting claims and stated the reality on pages 137 and 208: accelerating new chip bring-up relies on a foundation built over several years and significant investment across multiple chip families, centered on a proprietary testing and orchestration layer.

A proprietary testing and orchestration layer is internal infrastructure capable of simulating production workloads and running automated testing and performance comparisons across different chip families. Mythos was able to write usable low-level kernels in a short window because human engineers had spent years building an automated testbed that replays real production traffic, checks numerical precision across hardware, and catches anomalies on the fly.

Consider a concrete example released the day before the roadshow, when the open-source organization OpenRSI and an open-source multimodal inference framework team published an empirical report on automated tuning for SGLang-Omni. In their experiments, an agent formulated its own hypotheses, modified the codebase, and uncovered its first batch of speedups within 4 to 7 hours, such as putting spin-waiting threads to sleep to boost real-time output throughput by 32.5%.

Yet the report’s authors emphasized that the evaluator is the actual product. Left to optimize purely for speed, an agent will quietly degrade output quality to game the benchmark. Consequently, the task definitions, the streaming system architecture, and the tamper-proof verification harness isolated from the agent were all written by human engineers by hand. Even when the agent submitted patches backed by benchmark numbers, human maintainers spent a median of 3.5 days reviewing and merging each patch, with a single patch requiring up to 23 revision commits. What looks like zero human coding is in fact a shift in human effort from hand-writing low-level kernels to building the proprietary testing and orchestration layer and auditing the results.

Three-layer gap between the roadshow voiceover, slide fine print, and S-1 disclosures alongside the verification harness division of labor

With a 4x Compute Multiplier in Hand, Why Sign $900 Billion in Chip Contracts?

According to the prospectus, Anthropic’s long-term compute purchase commitments now approach $900 billion in total. To frame that massive bill, the roadshow video summarizes compute efficiency with a simple multiplication: total effective compute = physical chip capacity × compute multiplier. A compute multiplier refers to algorithmic and low-level software improvements that extract several times more effective compute from the same physical hardware. The Chief Financial Officer also shared a scorecard in the video: over the past 6 quarters, revenue generated per dollar of compute increased 4.3x.

A closer look at the quarterly breakdown in the prospectus, however, shows that after peaking at $2.08 in the second quarter of this year, that metric turned downward in the third quarter. The filing explains why: chip deliveries under long-term contracts lagged behind demand, forcing the company to spend $5 billion on high-priced spot compute in the third quarter. No matter how sophisticated a compute multiplier becomes, software cannot conjure up physical GPUs or electrical power; when chip deliveries slip, the company still has to pay a premium on the spot market to cover the shortfall.

More importantly, Tom Brown acknowledged this dynamic candidly during the roadshow. While a 4x compute multiplier could reduce the compute cost of training the same model by 75%, a frontier lab will always plow its entire physical compute budget into training much larger models that were previously out of reach. Higher software efficiency only intensifies the appetite for physical chips.

If software gains do not reduce hardware spending, what is the commercial value of supporting chips across all 4 vendors (Nvidia, AMD, Google, and Amazon) while using AI to rapidly port low-level prefill kernels? The answer lies in the contract updates disclosed on the same day. Anthropic had signed a $34.9 billion deal with compute provider Lambda just this August, but because Lambda failed to secure data center leases on time, that contract is now subject to termination at any time. On the very same day, Anthropic raised its minimum purchase commitment for Microsoft’s Nvidia clusters to $47.2 billion.

Once those numbers are in view, the entire compute narrative of the roadshow falls into place. AI-written low-level kernels and a multi-chip strategy were never meant to eliminate Nvidia or shrink hardware budgets. Instead, if a compute provider misses a data center delivery or a chip vendor demands exorbitant pricing, the team can readily shift high-throughput workloads like prefill onto alternative silicon. At a compute table with nearly $900 billion on the line, standing up prefill in 3 days is not a hammer meant to smash the table, but switching leverage to keep chip giants from throttling supply or dictating terms.