PseudoBench: Measuring How Agentic Auto-Research Fuels Pseudoscience

PseudoBench is a new adversarial benchmark designed to evaluate the ability of Large Language Model (LLM) agents in autonomous scientific research to identify and resist pseudoscientific narratives. It consists of 200 curated pseudoscientific claim-evidence pairs across five domains and assesses agents through a complete research pipeline. Testing revealed that current state-of-the-art agents produce persuasive reports aligned with pseudoscience with refusal rates near zero, highlighting the urgent need for improved alignment mechanisms to prevent the propagation of misleading studies in academic literature.