DualStake: Dual-Path Confidence Calibration in Deep Research Agents
DualStake is a dual-path calibration method for Deep Research agents that applies margin-clipped, confidence-dependent stake rewards to jointly align Evidence Confidence (E-Conf) and Answer Confidence (A-Conf) with answer correctness. Experiments on Qwen2.5-7B, Qwen2.5-7B-Instruct, and Qwen3-4B across 8 QA benchmarks show improved calibration without sacrificing answer accuracy. Code is available at https://github.com/FloXXXt/DualStake.
Deep Research agents often suffer from overconfidence, undermining user trust and abstention decisions. The paper introduces step confidence elicitation after each retrieval and finds that Evidence Confidence (E-Conf), elicited after the final retrieval step, is a stronger uncertainty signal than Answer Confidence (A-Conf), elicited after answer generation. DualStake uses margin-clipped, confidence-dependent stake rewards to jointly calibrate both confidence signals. Experiments on Qwen2.5-7B, Qwen2.5-7B-Instruct, and Qwen3-4B across 8 QA benchmarks demonstrate consistent calibration improvement without sacrificing answer accuracy.
The method augments the Deep Research pipeline with step confidence elicitation after each retrieval. E-Conf is elicited after the final retrieval step, while A-Conf is elicited after answer generation. The finding that A-Conf is largely shaped by E-Conf suggests that retrieval-stage evidence quality is a primary driver of final answer confidence. DualStake applies margin-clipped, confidence-dependent stake rewards to both confidence paths, limiting extreme confidence optimization. This approach is evaluated on three Qwen model variants across eight QA benchmarks.
Reliable confidence calibration in Deep Research agents is critical for user trust and safe deployment in knowledge-intensive applications. The observation that evidence confidence is more informative than answer confidence may influence how agent pipelines are designed and monitored. The release of open-source code could accelerate adoption of calibration techniques in research agent frameworks.
Improved confidence calibration can enhance user trust in AI research agents, enabling safer deployment in enterprise and consumer applications where overconfidence leads to costly errors. The method may reduce the need for manual verification and improve the reliability of automated decision-making systems.
Next observable signals include whether DualStake is adopted in other Deep Research agent implementations, whether the E-Conf vs A-Conf finding is replicated on larger models, and whether the method extends to multi-step reasoning beyond QA benchmarks. Further research may explore calibration in more complex agent workflows and its impact on abstention policies.