[2L4-GS-5c] Agents

2:45 PM - 3:00 PM JST(5:45 AM - 6:00 AM UTC)

[2L4-GS-5c-06] To What Extent Can LLM Reproduce Human Decision-Making?An Evaluation Using Social Survey Data

〇Hirotsugu OHBA¹, Shin-nosuke Ishikawa¹ (1. Rikkyo University)

Keywords:

Social Simulation,LLM Agents,Value Modeling

This study examines to what extent large language model (LLM) can reproduce human decision-making, using real-world data. Although recent approaches in social simulations generate human-like judgments by providing LLMs with demographic attributes and value-related profiles, their predictive validity has not been sufficiently evaluated empirically. We address this gap by conducting prediction experiments with social survey data collected in Japan and the United States. Partial information from multiple data groups— individual demographic attributes, MFQ2 (The Moral Foundations Questionnaire-2), responses to social moral dilemmas (“the Hardest Decisions to Sacrifice”), and personal moral dilemma questions —is given to an LLM, which is tasked with predicting the remaining judgments at the individual level. Comparisons across multiple input conditions indicate that, although the LLM does not uniformly reproduce actual responses, certain conditions suggest levels of agreement exceeding random chance but reflecting bias rather than prediction. This study provides foundational insights into the conditions under which the commonly assumed reproducibility of “human-like” decision-making holds when LLMs are used in social simulations, as well as the considerations that accompany such use.