CAVEWOMAN: How Large Language Models Behave Under Linguistic Input and Output Compression

The paper introduces Cavewoman, a two-channel evaluation protocol for assessing the effects of linguistic input and output compression on large language models (LLMs). It evaluates eight models across five datasets, revealing that output compression can reduce inference costs by 1.4-2.4x, while input compression typically increases costs by 1.15x on average, leading to longer, less accurate responses. This research highlights the importance of carefully managing compression strategies in LLM applications, as input compression may degrade performance and inflate operational expenses.