OPD-V: Visual On-Policy Self-Distillation with Modality Balance
Researchers introduced OPD-V, a visual on-policy self-distillation paradigm that uses modality balance as privileged information. A Positive Teacher with Zoom-In Image and a Negative Teacher with Mask Image reveal that modality balance affects reasoning correctness and token logits. Positive Modality-Balance Logits Margins define a Modality-Balance Trust Region to select on-policy tokens for self-distillation. Experiments cover 6 benchmarks and 4 models.
On-Policy Self-Distillation (OPSD) is a standard post-training method for improving visual reasoning in multimodal large language models (MLLMs). Existing approaches use privileged information from diverse inputs but overlook Modality Imbalance, where textual dominance prevents full multimodal integration. The paper constructs a Positive Teacher (Zoom-In Image) and a Negative Teacher (Mask Image) to show that Modality Balance itself can serve as privileged information. OPD-V leverages this by using Positive Modality-Balance Logits Margins to define a trust region for selecting on-policy tokens during self-distillation. The method is evaluated across 6 benchmarks and 4 models.
The key technical contribution is framing modality balance as a measurable signal for self-distillation. By comparing token logits from teachers with different modality imbalances, the method identifies tokens where the model effectively integrates visual and textual information. The Modality-Balance Trust Region filters on-policy tokens, potentially improving visual grounding without architectural changes.
This work addresses a practical limitation in deploying MLLMs: underutilization of visual inputs. If OPD-V proves robust, it could enhance visual reasoning in applications like document understanding, medical imaging, and autonomous systems, where balanced multimodal processing is critical.
Improved visual reasoning can increase the accuracy and reliability of multimodal AI products, reducing errors in vision-language tasks. This could lower deployment risks for enterprise applications in healthcare, legal, and manufacturing, and create differentiation for model providers.
Next signals include open-source code release, replication on larger models, and integration into popular MLLM training pipelines. If the approach generalizes, it may become a standard post-training step, similar to RLHF. Watch for benchmarks on real-world visual QA tasks and adoption by major AI labs.