Fisher-Guided Progressive Parameter Selection for Adaptive Fine-Tuning

The article presents FisherAdapTune, a parameter-efficient fine-tuning framework that utilizes Fisher-guided criteria to dynamically select parameter groups based on their curvature contributions over time. By employing a scale-invariant Jensen-Shannon distance to track Fisher distribution changes, the method aims to stabilize certain parameters while adapting others, enhancing performance on segmentation tasks with improved in-distribution and zero-shot transfer results. This approach is significant for practitioners as it offers a more adaptive and efficient strategy for fine-tuning pretrained models, potentially leading to better generalization in various applications.