论文

研究:搜索顺序塑造 LLM 多解谜题中的默认答案

Many Solutions, One Answer: Search Order Shapes the Default Solutions of Language Models

精选理由

模型遇到多解谜题总给同一个答案,这篇论文找到了原因:内部的搜索顺序。随机化输入顺序就能把答案广度提升 11 到 42 个百分点,代价几乎为零。做数据合成或想增加生成多样性的开发者可以直接套用这些技巧。

一篇 arXiv 论文研究了当约束谜题存在多个正确答案时语言模型的作答倾向,覆盖来自三个家族的四个开源权重模型,测试平台包括符号谜题和多解版 ZebraLogic。结果显示重复采样高度集中在同一个默认解上,远超均匀分布,且不同模型往往集中在同一个解。研究者推测这与模型内部的搜索顺序有关:在留出符号谜题上,比较按名称顺序访问实体与按初始候选最少优先访问实体两种搜索,后者在全部四个模型中多匹配 26 到 40 个百分点的输出。

正文

机制层面的干预实验证实了这一假设:颠倒取值顺序或显式指定搜索顺序,会把输出移向新顺序偏好的解。随机化呈现方式或提供随机搜索顺序,可将答案广度(有效解占比)在符号任务上提升 26 到 42 个百分点,在 ZebraLogic 上提升 11 到 26 个百分点,而有效性损失最多只有 3 和 4 个百分点。实验还发现随机化呈现比随机器选择指令更能拓宽答案分布。

这项工作解释了模型为何在多解题上趋同,同时给出低成本的缓解手段:调整提示中的呈现或搜索顺序即可显著增加答案多样性。对做多样化生成、数据合成和评测去偏的开发者有直接参考价值。

原文 · arXiv cs.AI

Many Solutions, One Answer: Search Order Shapes the Default Solutions of Language Models

When a constraint puzzle has several correct answers, language models tend to give the same one.We study four open-weight models from three families on symbolic puzzles and multi-solution ZebraLogic variants with exactly known solution sets.Repeated samples concentrate on one default solution far beyond uniform sampling, and different models often share it.Yet models usually produce even rarely sampled solutions when added clues uniquely identify them.On held-out symbolic puzzles, we compare visiting entities in name order with visiting those with the fewest initial candidates first. Both searches try values in the stated order. Where their pre-specified predictions differ, the latter matches a larger share of responses in all four models, by 26 to 40 percentage points.Reversing the stated value order or specifying a search order shifts responses towards the solution that the new order favours, on both testbeds. Randomising the presentation or supplying a random search order raises breadth (the effective share of solutions in use) by 26 to 42 percentage points on the symbolic task and 11 to 26 on ZebraLogic. The largest observed validity losses are 3 and 4 percentage points, respectively. Randomised presentation broadens answers more than a random-choice instruction in each of the three main models.

产业坐标与信源核验

本条情报归属 AITOP「论文」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:LLM、ZebraLogic、搜索顺序、多样性生成,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「arXiv cs.AI」(@Qinhan Hou, Jing Tang),经 AITOP 多维评分管线结构化提炼。

研究:搜索顺序塑造 LLM 多解谜题中的默认答案|AI热报