Learning When to Trust via Selective Context Preference Optimization
A paper titled 'Learning When to Trust via Selective Context Preference Optimization' was published on arXiv on 2026-08-06. It introduces MIST, a human-annotated benchmark for selective trust, and SC2W, a metric measuring how often misleading context flips correct answers. The paper proposes SCOPE, a method using Direct Preference Optimization over balanced preference pairs from four conditions (clean, misleading, correct-context, irrelevant-context) to reduce susceptibility to misleading signals while preserving accuracy with trustworthy context.
Researchers published a paper on arXiv introducing a method to improve language models' selective trust in external context. They created MIST, a benchmark with reasoning items under four conditions, and SC2W, a metric for susceptibility to misleading signals. Their proposed approach, SCOPE, uses Direct Preference Optimization on balanced preference pairs to reduce the rate at which misleading context causes errors, while maintaining performance when context is clean, correct, or irrelevant. The work argues that models should learn when to trust context rather than ignoring it entirely.
SCOPE addresses the failure mode of models becoming overly resistant to all context by balancing DPO training pairs across four conditions: clean-correct, misleading-wrong, correct-context, and irrelevant-context. This prevents the model from learning to ignore all context, instead teaching selective trust. The method mines failure cases where clean-correct answers become wrong under misleading context, and optimizes a standard DPO objective over these matched pairs.
This research highlights a practical challenge for AI systems that rely on external data: balancing robustness against misinformation with the ability to use helpful context. The approach could improve reliability in applications like retrieval-augmented generation, where models must discern trustworthy sources. Adoption may depend on integration into existing fine-tuning pipelines and demonstration of gains on real-world tasks.
Improving selective trust can reduce errors in AI applications that depend on external context, such as customer support chatbots, legal research tools, and medical information systems. This may lower risk and increase user trust, potentially accelerating enterprise adoption of context-augmented AI.
Next signals include potential open-source release of the MIST benchmark and SCOPE code, replication studies on larger models, and exploration of selective trust in multimodal or agentic settings. Industry may test the method in production RAG systems. Further research could extend the approach to dynamic trust calibration based on source reliability.