Plans Don't Persist: Why Context Management Is Load Bearing for LLM Agents

The paper introduces a framework for understanding context management in long-horizon LLM agents, particularly focusing on how plans are managed and their impact on performance. It presents the concept of replay pairing to diagnose the decay of plan signals in the hidden state of Llama-3.1-70B, revealing that plans do not persist as state and are heavily dependent on remaining in context. The findings indicate that naive eviction of plans can significantly reduce task success rates, underscoring the importance of effective context management strategies for practitioners developing LLM applications.