Pigeonholing: Bad prompts hurt models to collapse and make mistakes

The paper introduces the concept of "pigeonholing," where poor prompting leads to performance degradation and mode collapse in Large Language Models (LLMs). Through experiments with 10 models on various tasks, it was found that bad contexts can cause significant performance drops (38-40%) and narrow answer convergence, exacerbated by the number of conversation turns. To mitigate these effects, the authors propose a reinforcement learning approach with synthetic errors (RLVR), which improves model performance by 43-60% in adverse contexts, highlighting the importance of prompt design in LLM applications.