Can Editing 1 Neuron Fix Repetition Loops in LLMs?

The paper investigates the phenomenon of repetition loops in the Gemma 4 instruction-tuned models, where models exhibit high rates of looping (up to 95%) on long factual prompts. Through per-layer ablation and neuron attribution, the authors identify specific MLP neurons responsible for this behavior and demonstrate that targeted weight edits, including sign-inverted neuron modifications, can mitigate these loops without compromising general benchmark performance. However, the study also highlights that while these edits can reduce looping, they do not resolve deeper knowledge precision issues, indicating limitations in the approach for more complex generation tasks.