Clinical Term Extraction using Open-Source Small Language Models

The study evaluates the performance of 26 open-source small language models (SLMs) for extracting clinical terms relevant to amyotrophic lateral sclerosis (ALS) from unstructured clinical notes, using few-shot prompting without task-specific training data. The Qwen3-4B-Instruct-2507 model achieved the highest micro-F1 score, while a regex baseline outperformed all SLMs in overall micro-F1 and Hamming loss. The results indicate that hybrid extraction workflows combining SLMs with traditional rule-based methods may be more effective than relying solely on SLMs for clinical term extraction.