S-GBT: Smooth Growth Bound Tensor for Certified Robustness Against Word Substitution Attacks in NLP

The paper introduces the Smooth Growth Bound Tensor (S-GBT), a second-order method designed to enhance certified robustness against word substitution attacks in NLP models by bounding the Hessian element-wise. By incorporating a regularization term during training, S-GBT improves certified robust accuracy by up to 23.4% on benchmark datasets while maintaining competitive clean accuracy. This approach, applicable to LSTM and CNN architectures, emphasizes the importance of managing gradient variations alongside first-order sensitivity for developing more resilient NLP systems.