Improving Adversarial Transferability on Vision-Language Pre-training Models via Surrogate-Specific Bias Correction

The paper introduces DeBias-Attack, a novel method to enhance adversarial transferability in Vision-Language Pre-training (VLP) models by addressing surrogate-specific bias in adversarial optimization. This technique employs two perturbation branches: one optimizing on the original image and another on a weak-semantic image, enabling effective gradient correction and improving robustness against transfer-based attacks. The results demonstrate significant performance gains across various VLP models and tasks, highlighting its potential for practitioners focused on enhancing model robustness against adversarial threats.