LOCI: A Locator-Critic with Refinement Loop
LOCI is a training-free framework that decouples visual search from evidence verification using a Locator agent and a Critic agent in an iterative refinement loop. It improves accuracy on complex visual benchmarks for open-weight models like Qwen3-VL (+12.1 on V*, +5.8 on HR-Bench, +11.2 on VisualProbe-Hard) and proprietary models like Gemini 2.5 Pro (+8.9 on V*, +4.3 on HR-Bench, +4.8 on VisualProbe-Hard).
Vision-Language Models (VLMs) struggle on tasks requiring complex visual understanding due to failure to locate critical details. LOCI, a training-free framework, uses a Locator agent to propose candidate visual evidence and a Critic agent to evaluate relevance and sufficiency, iteratively refining evidence until adequate. This decoupled, self-correcting process achieves state-of-the-art results on multiple complex visual benchmarks, improving accuracy for Qwen3-VL and Gemini 2.5 Pro.
LOCI's decoupling of visual search from evidence verification allows iterative refinement without retraining, addressing flawed perceptual grounding. The reported gains on V*, HR-Bench, and VisualProbe-Hard suggest that locating critical details is a key bottleneck for VLMs, and agentic self-correction can mitigate it.
Training-free frameworks like LOCI can be applied to existing open-weight and proprietary models, potentially accelerating deployment of more reliable visual reasoning in applications such as medical imaging, autonomous systems, and document analysis without additional training costs.
LOCI offers a cost-effective way to improve VLM accuracy on complex visual tasks without retraining, potentially reducing error rates in high-stakes applications and enabling new use cases where precise visual grounding is critical.
Next signals include independent replication of LOCI's results, integration into popular VLM pipelines, and extension to other modalities or tasks. Watch for benchmarks that specifically measure visual grounding and for adoption by model providers seeking to improve complex visual understanding.