InnoEval: On Research Idea Evaluation as a Knowledge-Grounded, Multi-Perspective Reasoning Problem

InnoEval is a newly proposed framework for evaluating scientific ideas, designed to address the limitations of existing methods that often lack comprehensive knowledge and multi-dimensional assessment. It utilizes a heterogeneous deep knowledge search engine to gather dynamic evidence from various online sources and incorporates a diverse review board for consensus building across multiple evaluation metrics. Benchmark results indicate that InnoEval outperforms existing models in point-wise, pair-wise, and group-wise evaluations, aligning closely with human expert judgments, which is crucial for practitioners seeking robust evaluation mechanisms in AI-driven research.