LUQ: Layerwise Ultra-Low Bit Quantization for Multimodal Large Language Models

The paper introduces LUQ (Layerwise Ultra-Low Bit Quantization), a novel method for ultra-low-bit quantization (<4-bit) of multimodal large language models (MLLMs), addressing the challenge of high memory and computational demands. LUQ leverages the varying entropy of activations across transformer layers to selectively apply quantization, resulting in models like LLaVA-1.5 and Qwen-2.5-VL using 40% and 31% less memory than standard 4-bit models, with less than 10% performance degradation on multimodal evaluation benchmarks. This approach is significant for practitioners as it enables more efficient deployment of MLLMs without substantial loss in performance.