Geometry of Divergence: Tracking Hidden-State Trajectories for Adaptive Multi-Turn Reasoning
A research paper proposes tracking hidden-state trajectories of LLMs during multi-turn reasoning using temporal curvature and variance slope. Experiments across four tasks and three LLMs show these geometric signals distinguish correct from incorrect episodes before completion, and trajectory geometry can identify critical turns, increasing task success rates.
The paper formulates multi-turn reasoning as a hidden-state trajectory characterized by temporal curvature (directional consistency of turn-to-turn updates) and variance slope (expansion/contraction of exploration space). Across four tasks and three underlying LLMs, these signals distinguish correct and incorrect episodes prior to completion. The authors decompose episodes into three-action chains from four actions (Read, Write, Respond, Transfer) and find separability is action-dependent. Experiments demonstrate that trajectory geometry can identify critical turns, increasing task success rates.
The approach introduces two geometric signals—temporal curvature and variance slope—to monitor hidden-state trajectories in multi-turn LLM reasoning. These signals capture directional consistency and exploration-space dynamics, enabling early detection of representation drift. Action-chain decomposition reveals that different signals distinguish various chain patterns, suggesting action-specific monitoring could improve adaptive reasoning.
This research addresses a practical challenge for LLM agents: maintaining goal-consistent reasoning over long interactions under resource constraints. By identifying critical turns early, the method could reduce computational waste and improve reliability of multi-turn agent systems, potentially benefiting enterprise and consumer AI applications that rely on sustained dialogue.
The technique could lower inference costs by avoiding unproductive reasoning paths and increase task success rates in multi-turn AI agents, enhancing product reliability for customer support, coding assistants, and autonomous agents. It may create differentiation for platforms that implement adaptive reasoning controls.
Next observable signals include follow-up papers applying trajectory geometry to real-world agent frameworks, integration into LLM inference pipelines for early stopping or intervention, and benchmarks measuring success-rate improvements on multi-turn tasks. Adoption by major AI labs or open-source projects would indicate practical viability.