AnchorKV: Safety-Aware KV Cache Compression via Soft Penalty with a Refusal Anchor

The article introduces AnchorKV, a novel modification to key-value (KV) cache compression for large language models that enhances safety by incorporating a soft penalty mechanism in token retention. This approach leverages an offline safety anchor derived from a difference-of-means representation to bias token selection away from harmful prompts, thereby improving safety alignment without significantly sacrificing utility. This development addresses critical challenges in KV cache efficiency and security, making it relevant for practitioners focused on deploying LLMs in sensitive applications.