VIBE: A VAD-Informed Benchmark for Entity-Centered Affective Profiling of Large Language Model Outputs
Researchers introduced VIBE, a benchmark for entity-centered affective profiling of LLM outputs in Valence-Arousal-Dominance (VAD) space. The benchmark separates generation from external scoring, distinguishes scalar favorability, response-level VAD, and target-directed VAD, and reports profiles through an Affective Passport. Empirical findings show scalar favorability does not subsume arousal and dominance; valence is cross-validated (rV = 0.944 judge-human, rV = 0.954 inter-scorer), while arousal and dominance are single-scorer directional estimates.
VIBE is a new benchmark for evaluating how large language models affectively frame entities such as political figures, countries, and social groups. It uses Valence-Arousal-Dominance (VAD) dimensions to capture nuanced affective profiles beyond simple sentiment. The benchmark introduces a measurement contract that separates text generation from scoring and provides an 'Affective Passport' for each target. Initial validation shows high agreement on valence but highlights the challenge of reliably measuring arousal and dominance, which remain directional estimates.
VIBE's architecture decouples LLM output generation from VAD scoring, enabling standardized affective profiling. It employs a three-layer empirical design: scalar favorability, response-level VAD, and target-directed VAD. The high inter-scorer reliability for valence (r=0.954) suggests robust measurement, but the single-scorer nature of arousal and dominance indicates current limitations in capturing these dimensions precisely. This points to the need for multi-annotator setups or improved scoring protocols for arousal and dominance.
As LLMs are deployed in sensitive domains like news generation, policy analysis, and social media, understanding their affective biases toward entities becomes critical. VIBE provides a structured way to audit these biases, which could become a standard for responsible AI deployment. Companies developing LLMs may need to incorporate such profiling into their safety evaluations to mitigate reputational and regulatory risks.
VIBE offers a tool for enterprises and governments to assess LLM outputs for unintended affective framing, reducing brand risk and ensuring compliance with fairness standards. It could be commercialized as part of AI auditing services or integrated into model monitoring platforms, creating a new niche in the AI safety market.
Next signals include adoption of VIBE-like benchmarks in model evaluation suites, development of multi-scorer protocols for arousal and dominance, and integration of affective profiling into AI governance frameworks. Research may extend VIBE to multilingual and multimodal settings. If validated further, the Affective Passport could become a common reporting format for model transparency.