Human-Less LLM Serving: Quantifying the Human Tax on Throughput

This study quantifies the throughput loss in LLM serving systems due to human-centric latency metrics (TTFT and TPOT) when applied to long-horizon AI tasks that operate without human supervision. The research reveals that the "human tax" on throughput can range from 60-93% at 64K token contexts, particularly under high concurrency and tighter SLAs. The authors advocate for workload-class-aware SLA configurations to optimize performance for non-human tasks, suggesting that current serving systems may unnecessarily constrain throughput by uniformly applying human-focused metrics.