CoTAL: Human-in-the-Loop Prompt Engineering for Generalizable Formative Assessment Scoring and Feedback

The paper presents CoTAL, an innovative approach combining Chain-of-Thought Prompting and Active Learning for formative assessment scoring using LLMs, specifically GPT-4. It integrates Evidence-Centered Design to align assessments with curriculum goals and employs a human-in-the-loop method to refine prompts and rubrics iteratively, resulting in a scoring performance improvement of up to 38.9% over baseline methods. This framework is significant for practitioners as it enhances the reliability and quality of automated scoring systems in diverse educational domains, facilitating better feedback mechanisms for both teachers and students.