论文提出长期对齐框架:短期对话满意度或致认知萎缩
Alignment Myopia in Generative AI
这篇论文说现在的 AI 都在讨好你单次对话,长期可能让人变懒变依赖,作者给了个长期对齐的框架,做对齐和产品的人可以看看。
arXiv 论文《Alignment Myopia in Generative AI》指出,当前对齐方法依赖短周期反馈,只优化单次对话的即时满意度,忽略了人机长期互动带来的认知萎缩、社会侵蚀与人类自主性下降等纵向外部性。作者在人类福祉的四个维度上绘制了 AI 使用的短期效用与长期成本之间的张力,揭示现有奖励模型无法捕捉的结果。论文主张将偏好优化扩展为面向长期人类福祉的对齐策略,在多个时间尺度上进行权衡。
Alignment Myopia in Generative AI
Current generative AI alignment relies on a myopic paradigm that prioritizes immediate conversational satisfaction. By relying on short-horizon feedback, this approach ignores the profound, longitudinal externalities of persistent human-AI interaction, which can lead to adverse outcomes such as cognitive atrophy, social erosion, and diminished human agency. To address this, we propose a paradigm shift toward long-horizon alignment that accounts for how AI reshapes human experience over time. To this end, we map the tensions between near-term utility and long-term costs of AI use across four facets of human well-being to reveal outcomes that current reward models fail to capture. Ultimately, we advocate for moving beyond simple preference optimization toward alignment strategies that actively foster long-term human well-being by navigating these complex temporal scales.