Small Data, Big Noise: Adversarial Training for Robust Parameter-Efficient Fine-Tuning

The paper presents SDBN (Small Data Big Noise), a novel framework that integrates adversarial training with Parameter-Efficient Fine-Tuning (PEFT) to enhance robustness and generalization in NLP tasks, particularly when training data is limited. It introduces two variants: SDBN-h, which utilizes character-level edits for robust optimization, and SDBN-p, which employs LLM-generated variants, demonstrating significant performance improvements across benchmarks in low-resource scenarios. This work is crucial for practitioners as it addresses the challenges of noise and data scarcity in PEFT, enabling more reliable model adaptations without increasing parameter counts or computational demands.