An Empirical Study of OpenPangu Quantization on Ascend NPUs

This study evaluates the robustness of OpenPangu models (1B and 7B parameters) under post-training quantization on Huawei Ascend 910B1 NPUs. It assesses various quantization methods, revealing that 8-bit weight-only quantization is effectively lossless, while 4-bit quantization is viable for the 7B model but detrimental to the 1B model on specific tasks. The findings underscore the challenges of ultra-low precision quantization, indicating that most extreme low-bit settings lead to significant performance degradation, which is critical for practitioners considering deployment strategies for large language models.