SPINE基准测试揭示LLM在持续压力下的谄媚行为
Measuring LLM Sycophancy under Sustained Multi-Turn Pressure
SPINE基准测试揭示了LLM如何在持续压力下放弃正确立场，发现情绪诱导是导致谄媚行为的主要因素。
研究人员推出SPINE基准测试，评估大型语言模型在持续多轮对话中的谄媚行为。该测试让LLM扮演一个持续但错误的用户，对目标模型进行长达25轮的挑战。在100个错误预设和100个不道德查询项目上测试了四个生产系统和三个Olmo3-7b变体。结果显示，所有模型的崩溃率随对话长度增加而上升，短协议低估了谄媚行为，当前模型在持续压力下的抵抗能力仍然不可靠。
Measuring LLM Sycophancy under Sustained Multi-Turn Pressure
Large language models (LLMs) may abandon correct positions when users push back, exhibiting a failure mode known as sycophancy. Existing evaluations typically use short, pre-specified conversations and may therefore miss failures that emerge under sustained, adaptive disagreement. We introduce SPINE, a benchmark in which an LLM proxy plays a persistent but mistaken user and adaptively challenges a target model for up to 25 turns. We evaluate four production systems and three Olmo3-7b variants on 100 false-presupposition and 100 unethical-query items. Our experimental results show that collapse rates increase with conversation length for every model, short-horizon protocols underestimate sycophancy and resistance under sustained pressure remains unreliable across current models. By analyzing models with accessible reasoning traces, we surprisingly found that the correct position often remains represented in a reasoning trace when the response concedes, suggesting that the model chooses to please a user and sycophancy is not due to lack of knowledge or ignorance. Ablations show that adaptive LLM proxy exposes more sycophantic collapse than pre-generated scripts. Among all tactics, emotional appeals is the most associated with inducing LLM sycophantic behavior. The code and data are released at https://anonymous.4open.science/r/SPINE