Human-on-the-Loop Orchestration for AI-Assisted Legal Discovery

The paper presents a new approach to AI-assisted legal discovery, highlighting the issue of "trajectory collapse" in multi-step reasoning of autonomous LLM agents. It introduces a four-layer verification architecture encompassing planning, reasoning, execution, and uncertainty quantification to mitigate agentic failures, and demonstrates through simulation that implementing Human-on-the-Loop (HOTL) escalation can reduce privilege-waiver risk by up to 61% while minimizing the number of documents requiring attorney review. This work is significant for practitioners as it addresses critical reliability concerns in legal applications of LLMs, emphasizing the importance of human oversight in complex decision-making processes.