When Does Delegation Beat Majority? A Delegation-Based Aggregator for Multi-Sample LLM Inference

The paper presents a delegation-based aggregator called Propagational Proxy Voting (PPV), which outperforms majority voting in multi-sample LLM inference, achieving a +1.5 percentage point improvement on the MMLU-Pro benchmark. PPV utilizes within-group letter entropy and between-group reasoning geometry to inform its voting strategy, allowing it to effectively manage the weights of individual voters and enhance consensus without requiring gold labels or auxiliary training. This approach is significant for AI practitioners as it provides a more nuanced method for aggregating model outputs, potentially leading to better performance in unsupervised settings.