Prompt, Plan, Extract: Zero-Shot Agentic LLMs Workflows for Lung Pathology Extraction from Clinical Narratives

The study presents a zero-shot, agentic workflow for extracting lung pathology data from clinical narratives using five open-source generative LLMs. The best-performing model, GPT-OSS-20B, achieved a Micro-F1 score of 0.893, demonstrating competitive performance against a supervised GatorTron NER-RE baseline with a score of 0.960, while requiring no task-specific training. This approach highlights the potential of zero-shot LLMs to reduce the reliance on expensive manual annotation in medical data extraction, offering a scalable solution for practitioners in the field.