OpenBioRQ: Unsolved Biomedical Research Questions for Agents

The article introduces OpenBioRQ, a novel benchmark comprising 12,553 unsolved biomedical research questions across 12 domains, designed to evaluate the performance of agentic models in a retrieval-grounded setting. This benchmark uniquely focuses on the faithfulness and abstention of models when addressing open questions without predefined answers, revealing that even advanced models like Gemini-3-Pro, Opus-4.7, and GPT-5.5 only solve 29-60% of the hardest questions. The findings highlight significant challenges in model reliability and tool utilization, underscoring the need for improved methodologies in addressing complex biomedical inquiries.