SURGELLM: Rethinking Multi-Task Evaluation through Task-Aware Feature Gating with Class-Balanced Normalization

The article introduces SURGELLM, a unified transformer framework designed to enhance performance across diverse NLP tasks by addressing issues such as inductive bias mismatch and class-imbalance in feature statistics. Key innovations include a surgical feature gate, task-conditioned prefix tokens, and Instance-Weighted Normalization (IWN), which collectively improve macro-F1 scores, achieving 0.940 in benchmarks across four tasks. This framework is significant for practitioners as it provides a method to optimize multi-task learning by leveraging task-specific features and normalization techniques, potentially leading to more robust model performance in heterogeneous environments.