Self-Compacting Language Model Agents

The paper introduces SelfCompact, a novel scaffold for language model agents that allows models to autonomously determine when to compact their accumulated context during inference, addressing the limitations of fixed-interval compaction. This method combines a compaction tool for summarizing context with a lightweight rubric to guide its usage, resulting in significant improvements in efficiency and performance across six benchmarks, achieving up to 18.1 points better on competitive math tasks and 5-9 points on agentic search tasks while reducing token costs by 30-70%. This development is crucial for practitioners as it enhances the adaptability of LLMs in managing context without requiring fine-tuning or external supervision.