Multiagent Protocols with Aggregated Confidence Signals

The paper introduces three novel protocols for aggregating confidence signals in multiagent systems, addressing the lack of a method to produce a unified confidence measure for system outputs. By transforming and combining raw confidence signals through soft voting and a new Bayesian fusion technique, the proposed approach demonstrates improved discriminative ability (AUARC) compared to single-agent outputs and standard debate methods, while maintaining stable F1 scores. This advancement is significant for practitioners as it enhances the reliability of multiagent systems in NLP tasks, particularly in ambiguous scenarios.