DPT：用路径多样性奖励改进 LLM 并行思考
Parallel Thinking for Large Language Model with Path Diversity
这篇论文发现 RL 并行思考的路径会越长越像，DPT 靠惩罚前缀相似度让模型真正多路探索，代码也放出来了。
一篇 arXiv 论文指出，基于 RL 的并行思考常出现推理路径模式坍缩，多条轨迹退化为相似内容。作者提出 Diverse Parallel Thinking（DPT），通过惩罚并行路径的前缀相似度来奖励模型生成不同的推理策略。在多个推理基准上的实验显示，DPT 超越了此前的并行思考方法，取得 SOTA 成绩。代码已在 GitHub 开源（JiaruQian/DPT）。
Parallel Thinking for Large Language Model with Path Diversity
Parallel thinking has emerged as a promising paradigm for enhancing the reasoning capabilities of Large Language Models (LLMs) by exploring multiple reasoning paths simultaneously. However, we identify that parallel thinking via Reinforcement Learning (RL) typically suffers from the mode collapse of reasoning paths, where parallel thinking trajectories degenerate into similar ones. This homogeneity undermines the fundamental motivation of parallel thinking, that is, maintaining multiple hypotheses concurrently to prevent premature loss into a suboptimal solution. In this paper, we propose Diverse Parallel Thinking (DPT), a simple yet effective RL approach for parallel thinking designed to encourage path diversity. DPT rewards the model to develop distinct reasoning strategies by penalizing the prefix similarity of parallel thinking paths, thereby enhancing a more comprehensive exploration of the reasoning space. We conduct various experiments and analyses across multiple reasoning benchmarks to demonstrate the effectiveness of DPT. Results show that DPT successfully outperforms previous parallel thinking methods, achieving state-of-the-art performance over baselines. We validate path diversity as a key element to improve parallel thinking. Our code is available at https://github.com/JiaruQian/DPT.