Self-Referential Induction Increases Response Instability Relative to Unresolvable and Verifiable Questions in Large Language Models
A study measured response instability (1 - mean pairwise cosine similarity of sentence embeddings of compressed core claims) across 30 independent responses per question (360 total, Gemini API, temperature 0.7) for three question groups: self-referential prompts, unresolvable philosophical questions, and verifiable questions. Self-referential questions showed highest instability (0.343 ± 0.047), unresolvable philosophy questions intermediate (0.192 ± 0.008), and verifiable questions lowest (0.105 ± 0.058).
Research published on arXiv (cs.AI) on 2026-08-13 quantifies response instability in large language models across three question types. Self-referential prompts eliciting subjective-experience reports produce the most variable responses, while verifiable questions produce the most consistent. The study provides a quantitative baseline for induced subjective-experience reports in LLMs.
The methodology uses sentence embeddings of compressed core claims and cosine similarity to measure instability. The tight clustering for unresolvable philosophy questions (0.192 ± 0.008) suggests a distinct response mode compared to self-referential prompts. The high instability for self-referential questions indicates that first-person subjective reports are not stable across independent trials, which may reflect underlying stochasticity in self-modeling or lack of a fixed internal state.
This work highlights a potential reliability issue for applications that depend on LLM self-reports or introspective capabilities, such as AI safety evaluations, model interpretability, and user-facing agents that describe their own reasoning. The measured instability could inform benchmarks for consistency and trustworthiness.
For developers and enterprises, understanding response instability can guide when to rely on LLM outputs: verifiable questions are more stable, while self-referential or philosophical queries may require multiple samples or human oversight. This could affect product design for chatbots, mental health apps, and AI auditing tools.
Observable next signals include follow-up studies testing other models (e.g., GPT, Claude) and temperatures, attempts to reduce self-referential instability via prompting or fine-tuning, and integration of instability metrics into model evaluation suites. If instability persists across models, it may become a standard measure for subjective-report reliability.