MIDR: Enrichment-Augmented Indexing for Multimodal Document Retrieval
MIDR is a training-free framework for enrichment-augmented indexing that shifts multimodal reasoning to index time. During ingestion, a multimodal LLM converts rendered pages into verified textual fields indexed with BM25F and optionally fused with dense retrieval. On ViDoRe V3, MIDR Hybrid achieves 0.6219 average nDCG across five English domains, a 23.0% relative gain over BM25, remaining competitive with ColQwen2.5. On two French-document domains, enrichment bridges English queries and French page text, lifting BM25 from 0.1532 to 0.5448 nDCG and outperforming ColQwen2.5. Across all seven domains, MIDR leads ColQwen2.5 on four while using approximately 9x smaller index memory.
MIDR (Multimodal Indexing for Document Retrieval) is a training-free framework that addresses the representation problem in retrieval over visually rich documents. It shifts multimodal reasoning to index time by using a multimodal LLM to convert rendered pages into verified textual fields, which are indexed with BM25F and optionally fused with dense retrieval. On ViDoRe V3, MIDR Hybrid achieves 0.6219 average nDCG across five English domains, a 23.0% relative gain over BM25, and remains competitive with ColQwen2.5. On two French-document domains, enrichment bridges English queries and French page text, lifting BM25 from 0.1532 to 0.5448 nDCG and outperforming ColQwen2.5. Across all seven domains, MIDR leads ColQwen2.5 on four while using approximately 9x smaller index memory.
MIDR demonstrates that index-time multimodal enrichment can achieve competitive retrieval quality with significantly lower index memory compared to patch-level multi-vector approaches like ColQwen2.5. The framework's use of BM25F over verified textual fields enables text-centric serving while preserving multimodal grounding. The cross-lingual results suggest enrichment can effectively bridge query and document language mismatches.
The approach may reduce infrastructure costs for document retrieval systems by enabling text-centric serving with smaller indexes, potentially making multimodal retrieval more accessible for production deployments. The training-free nature of MIDR lowers adoption barriers compared to fine-tuned visual retrievers.
MIDR offers a cost-effective alternative for organizations needing high-quality retrieval over visually rich documents without the memory overhead of patch-level indexes. Its training-free design and cross-lingual capabilities may reduce development and infrastructure costs.
Observable next signals include adoption of MIDR in production retrieval pipelines, further benchmarks on additional languages and domains, and comparisons with other enrichment strategies. Potential extensions could explore dynamic enrichment strategies or integration with learned dense retrievers.