When Outputs Disperse, Does Epistemic Revision Follow? A Black-Box Coupling Diagnostic for Machine Collectives
A paper on arXiv proposes a black-box diagnostic to measure whether increased output dispersion in LLM collectives leads to genuine epistemic revision. It introduces a Coherence Index (CI) and Meta-Predictive Clarity System (MPCS) with a Re-Differentiation Protocol (RDP) to assess dispersion-revision coupling. The method is evaluated on five-agent collectives from two configurations.
The research introduces a diagnostic for LLM collectives that tests if interventions increasing output dispersion in embedding space also cause genuine revision of epistemic stance, rather than mere reformulation. It uses a Coherence Index to verify dispersion changes and per-turn stance annotation to measure revision. The MPCS with RDP is proposed as a reusable method for estimating this coupling regime, evaluated on five-agent collectives.
The diagnostic operates purely on generated text, making it black-box and model-agnostic. It separates output dispersion (measured by CI) from epistemic revision (measured by stance annotation), addressing the failure mode where LLMs produce diverse arguments without changing conclusions. The MPCS with RDP intervenes when outputs over-converge, potentially enabling more reliable collective intelligence in AI systems.
This work highlights a critical limitation in current LLM-based collective intelligence systems: apparent diversity may not reflect true epistemic diversity. For applications like multi-agent debate, collaborative problem-solving, or AI alignment, this diagnostic could become a standard evaluation tool to ensure collectives genuinely consider alternative viewpoints.
Organizations deploying multi-agent LLM systems for decision support, forecasting, or content generation could use this diagnostic to avoid false consensus and improve decision quality. It may also become a differentiator for AI platforms claiming robust collective reasoning capabilities.
Next signals include empirical validation on larger and more diverse model collectives, integration into multi-agent frameworks, and development of practical tools for monitoring epistemic health in deployed AI systems. The approach may influence research on AI safety and robustness by providing a measurable proxy for collective open-mindedness.