How do LLMs Evaluate Perceived Moral Agency? Investigating Moral Decision-Making in Human-Artificial Agents Interactions
A study compared how humans and LLMs evaluate perceived moral agency (PMA) across human and autonomous artificial agents in smart city scenarios. 190 human participants and various LLMs were tested using an adapted validated PMA scale. Humans were perceived as having higher moral agency than artificial agents. LLMs, when facing moral dilemmas, prioritized harm severity and contextual urgency over stable agent assessments, showing context-sensitivity.
This paper presents the first empirical study comparing human and LLM evaluations of perceived moral agency (PMA) for human and autonomous artificial agents in smart city scenarios. Using an adapted validated PMA scale, 190 human participants and various LLMs were assessed. Results show higher perceived moral agency for humans than artificial agents. In moral dilemmas, LLMs reason outward from the situation, prioritizing harm severity and contextual urgency over stable agent assessments, amplifying context-sensitivity.
The study adapts a validated PMA scale for LLM evaluation, revealing that LLMs do not maintain a stable moral agency attribution across contexts. Instead, their judgments are driven by situational factors such as harm severity and urgency, suggesting a context-dependent reasoning pattern rather than a consistent agent-based moral model.
As LLMs are deployed for moral advice and decision support, their context-sensitive moral reasoning may lead to inconsistent evaluations of the same agent across different scenarios. This has implications for trust and reliability in AI-assisted ethical decision-making, especially in smart city and autonomous systems contexts.
Understanding LLM moral agency evaluation can inform the design of AI systems for ethical advisory roles, risk assessment, and human-AI collaboration. Companies developing AI for smart cities, autonomous vehicles, or healthcare may need to account for context-sensitive moral judgments to ensure consistent and trustworthy behavior.
Future research may explore whether fine-tuning or prompting can stabilize LLM moral agency attributions, and whether such stability aligns with human moral psychology. Observing how LLM moral reasoning evolves with more embodied or autonomous agent interactions will be a key next signal.