Concept-Constrained Prompt Learning for Few-Shot CLIP Adaptation

The paper introduces Concept-Constrained Prompt Learning (CCPL), a novel framework for few-shot adaptation of CLIP that prevents overfitting by anchoring class prompts to frozen concept-level text prototypes. CCPL employs a text-space cosine consistency objective for alignment and includes concept dropout for regularization, achieving improvements in the base-to-new harmonic mean on DTD (+0.6) and EuroSAT (+2.9) benchmarks compared to CoOp, while maintaining performance on OxfordPets. This method is significant for practitioners as it enhances the adaptability of CLIP to new classes without retraining encoders, leveraging concept alignment for improved task performance.