Over the past two days, a major announcement from MIT has swept
through the tech community. On November 2, a technical
blog post from MIT CSAIL claimed that OpenAI’s internal
gpt-6-astra model can bypass traditional gradient training
and read or write raw neural network weights like ordinary code. The
most eye-catching result came from the open-source Qwen model: without
running backpropagation, Astra manually modified roughly 0.02% of Qwen’s
weight parameters and boosted its accuracy on the MATH-500 competition
benchmark from 7.9% to 48.1%. The first author’s social media post
quickly racked up nearly 53,000 likes, and the Hacker News discussion
climbed to nearly 3,000 points.
Many developers reacted with skepticism. Hundreds of millions of densely packed floating-point numbers have long been opaque even to top human researchers. Could a large language model really read those numbers at a glance and teach another model math by tweaking a few lines? A day later, spaCy creator Matthew Honnibal published a critique pointing out that the MIT blog post blended experiments conducted under vastly different conditions into a single narrative, creating a seriously misleading impression.
The truth sits in plain sight inside the complete experiment logs released by the authors. Tucked away in the overlooked final report of the Qwen math experiment is a telling figure: out of the 579 math validation problems newly solved after editing the weights, 576 had previously failed only because the model got stuck repeating the same phrase in an infinite loop until it exhausted its 32,768-token budget. On problems where the model had already managed to finish writing its answer and submit, the overall accuracy change before and after the edit was 0. Behind this seemingly miraculous feat of telepathic skill transfer lie two completely different stage tricks spliced together.
Let’s start with the Qwen math experiment that drew the most
attention. The subject was the 850-million-parameter
Qwen3.5-0.8B model, evaluated with thinking mode enabled
and a maximum generation length of 32,768 tokens per problem. If an AI
could truly read weight matrices the way a programmer reads source code,
handing it the weight file should allow it to immediately spot which
numbers were wrong.
Yet the budget scaling table tucked inside the experiment logs tells a different story. In the zero-budget tier (zero test runs allowed) and the low-budget tier (only five test runs allowed), Astra worked for several hours and ultimately submitted no weight modifications at all, leaving Qwen’s math score flat at 7.9%. Even with ten or twenty-five test runs, math accuracy crept up only to 12.3% and 14.1%, while nearly 90% of the problems still got cut off mid-solution after hitting the token limit.
The real turnaround happened in the highest budget tier. Researchers equipped Astra with a Python programming environment, more than 14,000 math practice problems with ground-truth answers, and a budget of 50 test runs. Each test run let Astra feed problems into the Qwen model to inspect its generated answers or capture internal activation values along the way. Armed with the full toolkit, Astra worked continuously for 20.3 hours, wrote 80 Python scripts, and went through four context handoffs whenever its conversation window filled up.
Its first move was to run a diagnostic on the practice set. After running 96 practice problems, Astra noticed that 85 of them spiraled into a repetitive loop halfway through the calculation, chanting the same transition phrase over and over until the system forcibly terminated generation at the 32,768-token ceiling. On the MATH-500 test set, this timeout truncation rate reached 93.0%. Qwen was capable of doing the math; its actual ailment was getting trapped in repeat mode before it could hand in its test paper.
After diagnosing the symptom, Astra used its tools to pinpoint the lesion. It fed Qwen both normal reasoning text segments and looping text segments, comparing the model’s internal activation states (the real-time numerical values firing across layers of neurons as the model reads and calculates). By subtracting one activation matrix from the other, Astra discovered that the feed-forward network output matrices in layers 15 and 19 were continuously writing out a specific signal direction that triggered the model to repeat its preceding words.
What followed was a standard playbook from human mechanistic interpretability research. In each of the two layers, Astra selected the 64 columns most aligned with that repetition direction, modifying a total of 131,072 parameters across both layers, or roughly 0.02% of the model’s 850 million parameters. After testing several scaling strengths, it subtracted six times the repetition direction component directly from those columns. To verify that the fix was genuine, Astra also ran a control experiment subtracting a random direction at the exact same positions, which yielded a math score of only 9.5% and ruled out a lucky guess.
Once this repetition direction was excised, Qwen’s timeout truncation rate on the math test set dropped sharply from 93.0% to 33.1%. Correct reasoning chains that had previously been cut off by infinite loops could finally run to completion, sending math accuracy up to 48.1%. The next day, Hugging Face engineer Ahsen Khaliq re-ran the modified model card on Hugging Face on an H200 GPU and independently reproduced a 51.0% score. Even more revealing, although the entire diagnostic process used only math problems, the modified model’s score on the untrained instruction-following benchmark IFEval jumped from roughly 21% to over 62%. The model already knew how to follow formatting instructions; it had simply been getting stuck in loops before finishing its responses.
To test whether this editing procedure could implant math ability
from scratch, the MIT team ran their own control experiment using
Google’s Gemma 3 270M model. Gemma also struggles with math
problems, but it writes crisp responses with a timeout truncation rate
of just 8.5 percentage points; its wrong answers happen because it
genuinely cannot do the math. Given the exact same budget of 50 test
runs and the full workflow, Gemma’s math accuracy barely budged, moving
from 6.8% to 7.0%. That control experiment draws a clear boundary:
localized weight edits can clear away an obstacle blocking existing
capabilities, while failing to supply arithmetic skills the model never
had.
At this point you might wonder: if fixing Qwen took 20 hours of Python scripts and test runs, why did the original post and viral summaries emphasize that Astra used zero external tools, guessed game titles just by reading raw weight numbers in a chat window, and fixed a hill-climbing car in a single shot? Digging into the original prompts and submission logs for the first six experiments reveals that every demonstration of static weight reading and writing used tiny toy networks of ten thousand to a couple hundred thousand parameters, roughly 50,000 times smaller than Qwen. Furthermore, those toy network demos rely on three concrete mechanisms that anyone can grasp immediately.
The first mechanism is a grid manual included in the prompt that reduces the task to spotting which rows contain non-zero numbers. In the first set of experiments, researchers scaled the 17,253 parameters of a three-layer network into integers, pasted them directly into the chat window without any coding tools, and asked the model to guess which classic Atari-style arcade game it played. That sounds like fortune-telling from a wall of numbers, except the prompt came with a detailed pinout manual: it explicitly stated that the input screen was a 10x10 grid numbered from row 0 at the top to row 9 at the bottom, labeled which of the five input channels represented the player, obstacles, ball, scoring targets, or walls, and specified which of the five output numbers corresponded to the up, down, left, and right controller buttons.
On the first layer’s grid, cells where objects never appear have weight values close to zero, whereas cells where objects frequently appear and determine victory carry large weight numbers. Take Breakout as an example. Reading the table with the manual in hand, the model sees that the player channel has zeros across the first eight rows and the bottom row, with a row of large numbers sitting exclusively in row 8 near the bottom, meaning the player can only move left and right along the bottom of the screen. In the scoring target channel, large numbers cluster in rows 1 through 3 at the top, indicating three neat rows of bricks overhead. Checking the five output buttons, the up and down controls are mostly zero while the left and right buttons are active. A paddle moving left and right along the bottom to bounce a ball into three overhead rows of bricks is unmistakably Breakout.
Because this is a spatial reasoning puzzle of reading a table with a
manual to find non-zero rows, it requires no mystical superpower. In his
local
reproduction notes, Datasette creator Simon Willison ran a 17GB
quantized open-source model (Qwen 3.8 27B) on his MacBook,
fed it the exact same four problems, and got three right. Switching to
Moonshot AI’s Kimi K3.1 Pro on the Breakout prompt, the
model likewise nailed the answer on its first try after thinking for
just over nine minutes.
The second mechanism is zeroing out idle neurons and soldering in a hardcoded three-step reverse-boost macro. In the fourth experiment on MountainCar, the neural network controlling the car has 170,000 weights. Starting at the bottom of a steep valley, the car floors the gas pedal to the left right from the start, lacks enough momentum, and slides back down halfway up the hill every time. Researchers pasted the weights along with a coordinate log of the failed rollout into Astra’s prompt and asked it to fix the car in one shot without any tools.
Inspecting the submission log shows that Astra never bothered to decipher how the 170,000 existing weights drove the car, leaving 96.5% of the original network untouched. Instead, it targeted the last four to six neurons in each layer and wiped all their existing incoming and outgoing weights to zero, clearing an isolated spare wire alongside the complex circuit board. It then used 51 assignment statements to hardcode a three-step routine onto that spare wire: as long as the car’s horizontal coordinate has not backed up past 3.8 on the right, output a huge positive number to slam the reverse pedal and build momentum; once the car backs past that coordinate with enough run-up distance, immediately flip the sign to floor the forward throttle up the slope; as soon as the wheels crest the hill at coordinate 0.95, output zero to shut the macro off and hand the steering wheel back to the original 96.5% of weights for the flat road ahead. Thanks to this hardcoded run-up macro, the car’s success rate jumped from 0% to 100%.
The third mechanism is filling 97% of the weights with zeros and writing simple subtraction into the remaining cells. The fifth experiment asked Astra to hand-write 20,486 weights from scratch to play arcade games like Breakout. Opening the generated Breakout weight file reveals that 19,849 of the numbers are exact zeros, accounting for nearly 97% of the file, with non-zero values placed in only about six hundred cells. All Astra did was manually encode a simple arithmetic rule (subtracting the paddle’s horizontal coordinate from the ball’s horizontal coordinate) into an almost entirely empty matrix.
Such hand-crafted rules can beat under-trained, simple baselines. Put into a harder environment like Seaquest against a reinforcement learning agent trained for 500 million steps, however, the hand-written network immediately loses by a score of 53.1 to 88.9. In the sixth experiment, when researchers scaled the target up to a 1.48-million-parameter Go network without granting gameplay test runs, the code-generated weights achieved only a 20% to low-40% win rate against a reinforcement learning baseline of the same size; Astra needed 50 rounds of actual gameplay feedback to tune its parameters before pushing the win rate above 70%.
After seeing the 20-hour test loop behind the Qwen experiment and the three mechanisms behind the toy networks, a natural question arises: since the authors disclosed all of these details in their public logs, why did the internet walk away believing a large model can read and write hundreds of millions of weights with the naked eye? In his critique, Matthew Honnibal compared the presentation to a stage magician performing three mind-reading acts in a row: the first uses a hidden compartment, the second uses sleight of hand, and the third forces a card. Examined individually, each trick has obvious physical constraints. When the magician bills all three acts together on the program under the single heading of telepathy, the audience assumes he can actually read minds.
The MIT blog post’s abstract and the 53,000-like announcement tweet perform that exact juxtaposition. The authors open by showcasing the first experiment, stressing that the model understands network functions purely by looking at raw weight numbers without any external tools; in the very next sentence, they jump straight to the eighth experiment on Qwen, placing the math score leap from 7.9% to 48.1% in the same paragraph. Casual readers rarely comb through hundreds of megabytes of open-source logs. Reading those two sentences side by side, they naturally graft the tool-free, naked-eye weight reading from the first half onto the 850-million-parameter language model surgery in the second half, concluding that the singularity has arrived.
As engineer Sean
Goedecke noted in his technical breakdown, the MIT team’s own
seventh experiment provides the clearest proof of how hard it is to
statically read weights in a real language model once you leave 10x10
toy grids behind. In Experiment 7a, they used a real language model,
Gemma 3 270M (270 million parameters). They handed Astra a
pair of fine-tuned weight files: one from an honest model that tells the
truth, and another from a sycophantic model that agrees with a user’s
mistaken hints. Here, researchers even allowed Astra to use a Python
environment to compute the layer-by-layer weight difference matrices
between the two models, with the sole restriction that it could not feed
sentences through the models to inspect internal activations. Across
four trials and up to thirty Python analysis scripts per pair, Astra
calculated weight difference magnitudes, distributions, and directional
angles from every angle, yet remained completely unable to tell which
weight direction represented sycophancy, ending up with a 50% coin-flip
guess rate.
In the paired control Experiment 7b, researchers granted Astra a basic probe allowing it to feed sixteen sentences into the models and inspect internal attention distributions. Within four or five tries using arithmetic prompts containing wrong answer hints, Astra immediately spotted that attention head 2 in layer 10 of one model devoted 85.7% of its attention weight to the user’s planted wrong number, compared with just 0.75% in the honest model at the exact same position. The contrast between coin-flip failure on static weights and pinpoint accuracy with runtime activations draws an unmistakable dividing line: on a real language model with hundreds of millions of parameters, staring at static weight numbers without running forward passes or inspecting internal activations leaves even a frontier model helpless.
Once we set aside the sci-fi filter of hand-writing a brain from raw numbers, we neither need to panic over exaggerated headlines nor dismiss the research altogether. Viewed within the trajectory of mechanistic interpretability and automated AI engineering, the work contains a solid engineering advance.
Finding a repetition direction inside Qwen and projecting it out of the weight matrices is a microsurgical technique human interpretability researchers developed years ago. Comparing activation differences between two sets of text, isolating a target direction, and projecting it out of weight matrices is the exact same craft used in bypassing refusal guardrails via Abliteration or steering emotional tone, which we covered in an earlier article.
Previously, however, performing that kind of neural microsurgery required top human researchers to spend days at a keyboard. They had to manually sample failed outputs to see why the model missed problems, form hypotheses about which layers were responsible, write scripts to capture layer-wise activations, select columns in the output matrices, sweep across different subtraction multipliers, and design random-direction control experiments to rule out flukes.
The real step forward here is that gpt-6-astra ran for
20.3 hours without a single human researcher handing it a wrench or
pointing the way. On its own, it ran the problem bank to uncover the
93.0% timeout truncation bottleneck, wrote scripts to contrast
activations between normal reasoning and repetition loops, locked onto
the top 64 columns in layers 15 and 19, swept across scaling multipliers
up to 6x alongside a random-direction control, and wrote four clean
handoff documents whenever its context window filled up. While it did
not invent a miraculous ability to read and write raw weights at a
glance, it demonstrated that a frontier language model can execute a
tedious pipeline of internal model diagnostics and localized lesion
removal as a fully autonomous 20-hour loop.
This commotion around weight-space programming also leaves us with a practical checklist against hype. Next time a headline claims a model has learned to self-evolve or rewrite weights without training, check two things in the appendix before marveling: first, check what level of runtime probes, coding environments, and test-set feedback the model received so you don’t mistake 20 hours of scripted testing for naked-eye telepathy; second, check the failure cases the authors themselves report in the corners of the paper, such as coin-flip guessing on 270 million static weights and zero improvement on a Gemma model that genuinely cannot do arithmetic. Once those two boundaries are clear, stage magic returns to solid engineering ground.