Keyless Attention: Value-Space Routing and Value-Only Caching for Efficient Transformers

The article introduces Keyless Attention, an innovative mechanism that eliminates key projections, relying solely on queries and values. This approach results in a 50% reduction in KV cache memory and access overhead compared to traditional attention, while maintaining or exceeding decode throughput. Keyless Attention also implements Depth-$m$ Attention Factorization, achieving competitive perplexity scores across multiple models (e.g., GPT-2, Pythia, Qwen2, Llama) and demonstrating superior performance on commonsense reasoning benchmarks, making it a valuable advancement for practitioners focused on efficiency in transformer architectures.