AI repairs AI without training: an edit to 0.016 percent of the weights. What it means for open models

TL;DR. On November 2, researchers at MIT showed that one AI model can repair another directly in its weights, without any training. OpenAI's GPT-6 Astra found the cause of endless thinking loops in the small open model Qwen3.5-0.8B and changed 0.016 percent of its weights to fix it. Afterwards the model solved 48.1 instead of 7.9 percent of the problems in a math test. We downloaded both versions and compared them number by number: from the outside they look the same, only the checksum gives the difference away. Open models can now be repaired with precision, and changed just as quietly. If you run them in-house, you need to know which version is running.

What the researchers did

A language model consists of hundreds of millions to trillions of numbers, its weights. They are normally created and changed through training: lots of examples, lots of computation steps, lots of computing power. In their post "Sparks of Weight-Space Programming", the team around Omar Khattab at MIT CSAIL tried something else. They gave GPT-6 Astra access to the weights of another network and a task in plain words. Training was not allowed. Astra could only read, calculate and replace individual numbers.

Across eight experiments, Astra recognized from raw weights which game a network had been trained on, repaired a vehicle controller that had failed all 256 test runs, and wrote small game-playing networks from scratch. The biggest effect came in the eighth experiment, with Qwen3.5-0.8B, a small open language model from Alibaba. It has a known weakness: while thinking, it gets caught in loops. Its model card warns about this itself.

One line, 569 times

The published test answers show what these loops look like. Asked to convert the point (0, 3) to polar coordinates, the original model wrote the same line 569 times in a row until the answer was cut off at 32,768 tokens. There was no result.

Astra searched the weights for the spot that triggers these loops. It found a direction that two components in layers 15 and 19 write into the model, and removed it there from 64 columns each. Nothing more. Afterwards the model solved 48.1 instead of 7.9 percent of math competition problems (MATH-500) and 67.1 instead of 13.8 percent of word problems (GSM8K). Following instructions also rose from 21.0 to 62.3 percent, because the loops do not only occur in math.

At Hugging Face, the edited model was independently re-run the same day, with 51.0 percent on MATH-500. On the same 25 problems that were also run there with the original, it is 47 to 5 correct answers out of 100 each.

Our measurement: what changed in the file

The edited version is publicly available on Hugging Face. On November 3 we downloaded it together with the original, 1.75 gigabytes each, and compared them number by number.

• Tiny. Of 873,438,784 numbers in the file, 130,188 are different, 0.015 percent. The researchers arrive at 0.016 percent because they only count the language part.
• Invisible from the outside. Same components, same names, same settings. You cannot tell from the file what the change does.
• Measurable at once. The checksums (SHA-256) of the two files are completely different. The comparison takes seconds.

Everything is open here: the researchers publish every round, every calculation step and every change. That it can be done differently was shown back in 2023 by the security firm Mithril Security with PoisonGPT: an open model, changed on purpose so that it spreads a false fact, yet only 0.1 percent off the original in a standard test.

The catch

• A blog post, not a peer-reviewed paper. One team, one editing model, one run per setting. So far the result has been re-run, not the method.
• A lot of effort for one edit. The best run took 20.3 hours, 4.7 of them spent waiting out an outage, with 158 rounds, 80 calculation runs and 50 trial runs of the model, at the highest reasoning level. What it cost is not stated anywhere. A comparison with ordinary fine-tuning is missing.
• With fewer trial runs it barely worked. With 10 or 25 trial runs, MATH-500 only reached 12.3 and 14.1 percent.
• Repaired, not made smarter. With Gemma 3 270M, a model without loops, math rose from 6.8 to at most 7.7 percent. The researchers write themselves: the edit could remove an obstacle, but not add a missing ability.
• The technique itself is not new. Removing a behavior from a model along a single direction has been known since 2024, for refusals for example. What is new is that a model finds the spot by itself. Qwen's own setting against repetition was, by the way, already switched on in the tests.
• Reading does not always work. Astra could not tell from the weights alone, in four attempts, which of two models tells users what they want to hear. It only succeeded once it could look at the running model.

Critics such as the PauseAI initiative consider this direction dangerous, because models could use it to improve their successors. That has not been shown: the researchers did not demonstrate a model that improves itself round after round.

What this means for mid-sized companies

If you use AI through ChatGPT, Copilot or an API, you will not notice any of this for now. The weights stay with the provider. It is different with open models that companies run themselves, for example because data must not leave the building. There, the work has two sides.

The good side: a known weakness of a small model may soon be fixable without training data and without a graphics card, with an edit that leaves the rest of the model untouched. The researchers also hope to find hidden backdoors directly in the weights one day.

The other side: a change to 0.015 percent of the numbers cannot be seen from the outside. The easier such edits become, the more it matters where a model file comes from and whether it is still the one you tested.

What you should do now

1. Get open models only from the original source. From the publisher's own account, not from a copy with a similar name.
2. Pin the version. For every model file you use, record the version and checksum and compare them before starting.
3. Treat every change as a new version. Whether fine-tuned or repaired: test it with your own test cases before use, record the result and keep a way back to the old version.
4. Hold off on your own edits. For now this is research with one model and one team. It becomes interesting for production once there are tools that make every edit traceable and checkable.

An AI finding a fault in another AI and removing it with precision is new. For your business, today it mostly means this: you should know which model file is running on your systems. If you want to set this up properly for your own AI applications, we are happy to help.