HyperQuant: A Rate-Distortion-Optimal Quantization Pipeline for Large Language and Diffusion Models

HyperQuant is a new post-training quantization pipeline designed for large language and diffusion models, achieving optimal rate-distortion performance. It demonstrates superior results compared to existing methods like HIGGS, TurboQuant, and OCTOPUS, with weight compression of approximately 3.9x and KV cache compression of 3.79x at 4 bps while maintaining near-lossless quality. The pipeline integrates a Randomized Hadamard Transform, low-dimensional optimal lattice quantization, and advanced coding techniques, making it relevant for practitioners aiming to enhance model efficiency without sacrificing performance.