Wrong Prediction, Right Answer: Recovering Evidence from Collapsed LLM Sequence Scores
A study on arXiv (cs.AI) reports that hidden-state probes can decode correct answers even when native sequence scoring collapses in large language models. A diagnostic protocol using a minimal, target-label-free additive correction, fitting two parameters on as few as 25 unlabeled examples, recovers 9-34 accuracy points for Qwen3.5 models and transfers to OLMo-2-1B and Llama-3.1-8B. Recovered decisions persist on hard instances and exceed count-preserving permutation baselines.
The paper argues that apparent zero-shot reasoning deficits in LLMs may be expression failures masking intact internal logic. It introduces a method to recover correct answers from collapsed sequence scores using a simple additive correction, demonstrating significant accuracy gains across multiple models.
The method leverages hidden-state probes to extract correct answers despite collapsed sequence scoring. The additive correction is minimal and target-label-free, requiring only two parameters and 25 unlabeled examples, suggesting a low-cost diagnostic for latent reasoning capabilities.
This research implies that benchmark evaluations may underestimate LLM reasoning abilities. It could lead to new evaluation protocols and diagnostic tools for model developers, potentially shifting how capabilities are measured and reported.
For AI labs and enterprises, this could improve model assessment and reduce false negatives in capability testing, potentially guiding fine-tuning or prompting strategies to unlock latent performance.
Next signals include replication studies on larger models, integration of the correction into evaluation frameworks, and exploration of whether similar readout gaps exist in other tasks like planning or tool use.