P-Check: Advancing Personalized Reward Model via Learning to Generate Dynamic Checklist

P-Check is a new personalized reward modeling framework that introduces a dynamic checklist generator for evaluating user preferences, addressing the limitations of static user context in existing models. It employs a Preference-Contrastive Criterion Weighting strategy to prioritize evaluation criteria based on their relevance to individual judgments. The framework shows improved reward accuracy and performance in downstream personalized generation tasks, particularly in out-of-distribution scenarios, making it significant for practitioners focused on enhancing user alignment in AI systems.