KnowHal: A Knowledge-Driven Benchmark for Comprehensive Multimodal Hallucination Evaluation
KnowHal is a benchmark that evaluates multimodal hallucination across four dimensions: entity, attribute, relation, and knowledge. It contains 1,800 samples across 10 domains and 50 categories, constructed via a semi-automated pipeline using LLM assistance, CLIP-based filtering, and human verification. Fourteen representative MLLMs were evaluated, and the knowledge dimension was found to be the most challenging for nearly all models.
Researchers introduced KnowHal, a benchmark designed to unify the evaluation of hallucinations in Multimodal Large Language Models (MLLMs) by explicitly including a knowledge dimension alongside entity, attribute, and relation hallucinations. The benchmark comprises 1,800 paired positive and negative questions over shared images and entities, enabling controlled comparisons of perceptual errors, knowledge-related errors, and false-premise acceptance. Evaluation of 14 MLLMs revealed that knowledge-related hallucinations pose the greatest difficulty, highlighting a critical area for improvement in trustworthy AI.
KnowHal's design uses paired positive/negative questions over identical images and entities to isolate knowledge errors from perceptual ones. The semi-automated pipeline leverages LLMs for question generation, CLIP for filtering, and human verification for quality, ensuring scalable yet reliable benchmark construction. The finding that knowledge hallucinations are the hardest dimension suggests current MLLMs struggle with integrating external or implicit knowledge beyond visual recognition.
The benchmark addresses a gap in evaluating MLLM trustworthiness, which is crucial for deployment in high-stakes applications like medical imaging, autonomous driving, and content moderation. By highlighting knowledge hallucination as a key weakness, it may drive research and investment in retrieval-augmented generation or knowledge-grounded architectures for multimodal systems.
For AI developers, KnowHal provides a diagnostic tool to identify and reduce knowledge errors, potentially improving product reliability and user trust. Enterprises deploying MLLMs can use such benchmarks to select models with lower hallucination rates, reducing risk in customer-facing applications.
Expect follow-up work on mitigation strategies for knowledge hallucinations, such as improved training data curation or hybrid neuro-symbolic approaches. The benchmark may become a standard evaluation tool, influencing model development roadmaps. Watch for industry adoption of similar evaluation frameworks in model cards or regulatory compliance.