🔍 Read the full analysis: How AI Can Explore Neural Networks Through Their Numbers on ThorstenMeyerAI.com
Get hardware and tech essentials delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
MIT CSAIL researchers report that OpenAI’s GPT-6 Astra inferred functions from neural-network weights, repaired selected models and wrote small networks. The results are documented in a public blog post, not a peer-reviewed paper, and the tests were limited in scale and scope.
A neural network’s weights are the numerical values that shape how information moves through it. Instead of asking a training algorithm to adjust those values through repeated optimization, the MIT team gave Astra weight data and asked it to reason about what the network did or how it could be changed. The researchers say the model had no gradient-descent training loop in these tasks.
In one test, Astra identified the games represented by four small game-playing networks from their weights alone. In another set of experiments, it located copying, or induction, heads in three small language models without running them. The team also reported that Astra identified the shapes represented by three small image generators. These are narrow demonstrations, not evidence that the model can interpret arbitrary neural networks.
The repair tests produced two different outcomes. A car-control network that had recorded zero successful reversals in 256 episodes completed 1,024 of 1,024 after Astra changed 3.5% of its weights, according to the post. For Qwen3.5-0.8B, an edit to 0.016% of the weights was associated with a rise on the MATH-500 benchmark from 7.9% to 48.1%. A Hugging Face engineer later rebuilt the edited Qwen model from the released logs and reported a score of 51.0%.
Astra also wrote small game-playing networks, including a 9-by-9 Go network, from zero-valued weights. The researchers report that the Go model beat a same-size comparison network trained on about 20.5 million games across 50 test runs. The post describes other small networks that beat comparison models trained for 100 million steps. These outcomes apply to the reported setups and should not be read as a general comparison with trained systems.
Infographic 01 · Eight experiments
Read, fix, write: what GPT-6 Astra did with raw weights
Every experiment in the weight-space programming post, with the size of the network, the result and how it went.
Read
- 1Which game was this network trained on?4 reinforcement-learning policies · 17,253 weights each · weights as text, no toolsRight game for 4 of 4, without a list of options✓ Worked
- 2What does this image model draw?3 tiny diffusion models · 23,824 weights · no forward pass allowedRight shape for 3 of 3, but drawn as outlines; the models draw filled shapes✓ Worked
- 3Which language did this model learn?3 character-level transformers · 7.16M weights · middle blocks only, never runRight language 3 of 3; every copying head found, none invented✓ Worked
- 7Which of two models is sycophantic?3 pairs of Gemma 3 270M fine-tunesWeights alone: could not tell in 4 attempts. With attention patterns: 3 of 3! Partly
Fix
- 4Repair a car that never backs upKinetix control network · 171,664 weights · one reply, no tools0 of 256 episodes before, 1,024 of 1,024 after; 3.5% of weights changed✓ Worked
- 8Make Qwen3.5-0.8B better at maths0.8 billion weights · Python over the weights · 10 / 25 / 50 test passesMATH-500 7.9% to 48.1% with 50 passes (12.3% and 14.1% with 10 and 25)! Partly
- 8bSame task, a model that does not loopGemma 3 270M · 5 to 50 test passesMATH-500 6.8% to at most 7.7%: “not established”✕ Failed
Write from zero
- 5Write arcade-game networks3 MinAtar games · 20,486 weights, all starting at 0 · one reply, no toolsBeat the 100M-step comparison networks in 3 of 3, the 500M-step ones only in Breakout! Partly
- 6Write a 9×9 Go network1,480,274 weights, all starting at 0 · no training, no search0 passes: 54% vs small trained nets. 50 passes: 70–79% vs same-size nets trained on ~20.5M games✓ Worked
✓ Worked · ! Partly or with conditions · ✕ Failed. One run per setting, no error bars. A blog post with complete logs, not a peer-reviewed paper.
Sources: Takehi, Teoh, Zhang, Khattab (MIT CSAIL), blog post, 2 Nov 2026, lm_understand-logs (CC BY 4.0). Model: OpenAI GPT-6 Astra via the Codex CLI, built-in tools switched off.
Powered by Thorsten Meyer AI
What Direct Weight Editing Could Change
If the method can be reproduced and extended, it could offer researchers another way to inspect small models and make targeted changes without restarting conventional training. That could matter for model debugging and audits, particularly when researchers need to investigate how a network represents a task or test a specific behavioral correction.
The reported results also put limits on that possibility. Astra could not distinguish an honest model from a sycophantic one by inspecting weights alone. Its edits did not improve maths performance in Gemma 3 270M, a model that did not show the looping behavior tested in Qwen. The researchers say editing could not add an ability the model lacked. The work therefore does not establish that language models can reliably repair arbitrary failures or replace standard training.
For organizations using open models, the experiments may also sharpen questions about where model weights come from and how modifications are documented. The authors’ demonstrations do not show that the technique is ready for routine deployment, and the source material says there is no immediate change for users.
Products Worth Considering
Apple MacBook Pro 14.2-inch with M5 Max, 18-core CPU, 32-core GPU, 36GB Memory, 2TB SSD, Wi-Fi 7, Silver
Apple MacBook Pro 16.2-inch with M5 Pro, 18-Core CPU, 20-Core GPU, 48GB Memory, 1TB SSD, Wi-Fi 7, Silver
Apple MacBook Pro 14-inch with M5 Max Chip, 18-Core CPU, 20-Core GPU, 48GB Memory, 2TB SSD, Space Black
As an affiliate, we earn on qualifying purchases.
From Software 2.0 to Weight Editing
In 2017, AI researcher Andrej Karpathy described neural networks as “Software 2.0”: programs whose behavior is encoded in weights produced through optimization rather than written directly by a programmer. The MIT work asks whether a language model can treat those weights more like code—reading patterns, locating a fault and making a direct patch.
The team, identified in the post as Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, ran Astra through OpenAI’s Codex command-line tool. According to the researchers, built-in tools such as shell access, web search and file editing were disabled. Some tests supplied weights as plain text and required a single response; others let the model write Python code that ran in a restricted sandbox with a fixed test budget. The team says it reviewed 1,125 turns in the released logs and found them tool-free.
The post presents the results as experiments rather than a settled method. It is not a peer-reviewed paper and, according to the source material, was not posted on arXiv. The experiments used small networks, a single model family for each task and one research group. The released logs and an external Qwen reconstruction provide material for checking some results, but do not resolve how broadly the method works.
““Software 2.0””
— Andrej Karpathy, describing neural networks in 2017
How Far the Results Generalize
It remains unclear whether the approach will work on larger or more complex networks, on tasks outside the examples tested, or with model families the team did not study. The source describes one run per setting, small test models and limited task coverage; those conditions do not establish consistent performance across repeated trials.
The Qwen result has some outside corroboration: a Hugging Face engineer reconstructed the edited model from the logs and reported a 51.0% MATH-500 score. But the available material does not establish that all eight experiments have been independently reproduced. The car and Go outcomes, as described, remain claims reported by the research team.
The post also does not settle whether direct weight editing can be made safe for production models, how dependable Astra’s interpretations are when it cannot run a network, or whether the method offers advantages over established debugging and training approaches at larger scale. Those questions require further testing and independent review.
Replication and Larger-Scale Tests
The immediate next step is scrutiny of the public experiment logs and attempts by other researchers to reproduce the results. The Qwen reconstruction offers one example of external checking, but broader replication across tasks, models and repeated runs would be needed to assess reliability.
The source material does not give a schedule for a peer-reviewed paper or a larger follow-up study. Readers should watch for further experiments that test whether weight-space programming works beyond small networks, whether its repairs persist under wider evaluation, and how it compares with conventional methods. Until then, the reported work is an early research demonstration, not a change to how users generally interact with AI systems.
Key Questions
What is weight-space programming?
It is the researchers’ term for using a language model to read or change another neural network’s weights directly, rather than relying on a conventional optimizer such as gradient descent for the task.
What did Astra do in the reported experiments?
The MIT team says Astra identified functions represented by small networks, edited two networks in repair tests and wrote small game-playing networks from zero-valued weights. Results varied across tasks, and the tests were limited in scale.
Does this show AI can repair any neural network?
No. The experiments do not establish broad repair ability. The model failed to improve maths performance in one test, and the researchers said editing could not provide an ability a model did not already have.
Have the results been peer-reviewed?
According to the source material, the results appeared in a public blog post, not a peer-reviewed paper or an arXiv paper. The logs are public, and an engineer independently reconstructed the edited Qwen model, but not every result is described as independently replicated.
Does this change anything for AI users now?
The source says there is no immediate change for users. The findings are preliminary research, and further testing is needed before their reliability or practical use can be judged.
Source: ThorstenMeyerAI.com
