DrugBench: Evaluating AI Control Protocols for Medication Harm Mitigation

DrugBench is a newly introduced evaluation benchmark designed to assess AI control protocols aimed at mitigating medication-related harm in clinical settings. It comprises 3,671 multi-turn medical conversations integrated with drug information from FDA labels, focusing on four key harm categories: drug interactions, contraindications, dosing constraints, and patient action restrictions. This work emphasizes the need for severity-based monitoring in safety protocols, highlighting that existing methods can be vulnerable to subversion, which is critical for practitioners deploying AI in healthcare to ensure patient safety.