Repair Without a Repairer: Five Mechanisms Called Self-Repair in Neural Networks, Why Much of the Measured Restoration Is Restored Scale, and What Would License the Claim That a Network Restores Its Own Performance

Abstract

"Self-repair" is used in at least three research communities that do not cite one another. In transformer interpretability it names the Hydra effect: when one component is ablated, later components shift their output and partly restore the logit. In neuromorphic engineering it names astrocyte-inspired plasticity that restores the accuracy of a spiking network after synapses are deleted. In model editing, unlearning and pruning it names a deliberate weight update, or, with the sign reversed, the unwanted return of removed knowledge. In this analysis the author reviews and synthesises published results to show that these uses name five mechanisms, not one, and that they differ on four things a repair claim has to specify: the fault model, the quantity restored, who does the repairing, and whether the response depends on the fault having happened. Read side by side, the literatures share a regularity that neither has stated: a substantial part of what each measures as restoration is the return of activation or weight scale, not the recomputation of lost content. In one transformer study, LayerNorm rescaling accounts for roughly 30 percent of an ablated head's direct effect, although two later studies that hold the normalisation fixed or bound it find that it does not itself produce the backup or counterweight response; a plain weight renormalisation recovers most of the accuracy restored in an astrocyte study; range restriction, a low-cost hardware defence, works by the same route. A 2026 result reports that the transformer response follows a fixed affine law that is already present before any ablation, which makes it a counterweight rather than a repair. Language models that absorb the loss of whole heads or layers can be crippled by zeroing one weight, and an image network that shrugs off random bit flips is destroyed by a handful of targeted ones, so robustness is a property of a network under a stated fault model, not of the network. The paper consolidates the measured values, gives a five-condition test for the phrase "a network that restores its own performance", finds that only plasticity-based and search-based mechanisms can meet it, and names five studies that would settle the open parts.

About the author

Pranay Mahendrakar is an ai specialist and large language model engineer based in Bengaluru, India. He builds production artificial intelligence systems and publishes open-access research on how those systems fail. See all 85 papers by Pranay Mahendrakar, or his ORCID record.