用 2022 EU Loneliness Survey 检验 LLM 能否复现孤独感年龄差异
Do LLMs Reproduce the Age Gap in Loneliness? Evaluating Survey Simulation against European Survey Data
拿四个 LLM 模拟欧洲四国调查，看谁能复现真实的年龄差距，结果三个模型翻车，适合做问卷模拟的人看看。
研究以 2022 EU Loneliness Survey 为基准，比较法国、意大利、波兰和瑞典 18-34 岁与 65 岁以上人群的孤独感差异。在指定年龄的模拟中，Mistral Small 4 和 GPT-5.6 Terra 分别在三国和两国逆转了年轻人更孤独的真实排序，主因是高估老年人的孤独均值。DeepSeek V4 Flash 在所有四国都保住了排序，但整体低估两组均值。直接分布估计在 Mistral 和 Terra 上比模拟有更低的年龄差距误差。
Do LLMs Reproduce the Age Gap in Loneliness? Evaluating Survey Simulation against European Survey Data
Large language models (LLMs) are often used to simulate survey respondents, but matching survey averages does not establish accurate demographic differences. We test whether LLM outputs reproduce within-country age gaps in loneliness using the 2022 EU Loneliness Survey, comparing adults aged 18-34 and 65 or older in France, Italy, Poland, and Sweden. We evaluate simulation with and without age information and direct distribution estimation against a probability-based sample, with a non-probability sensitivity reference. Both human samples show higher mean loneliness among younger adults. Under age-specified simulation with standard response-option order, Mistral Small 4 and GPT-5.6 Terra reverse this ordering under all three prompt framings in three and two countries, respectively, mainly by overestimating older-adult means. DeepSeek V4 Flash preserves the ordering everywhere but underestimates both group means, with errors largely cancelling in France and Italy. Against the probability-based reference, direct estimation has lower absolute age-gap error than simulation for Mistral and Terra in every country. For DeepSeek, simulation has lower gap error but direct estimation has lower group-mean error; these comparisons depend on low-validity older-adult simulation cells. Evaluating LLM outputs as proxies for age groups within countries therefore requires checking both group means and their contrast against human survey data.