思维链文本的可读性不等于可解释性，研究揭示了LLM判断推理步骤重要性的局限性。
该论文研究了思维链模型推理步骤的重要性判断问题。研究通过蒙特卡洛回滚方法估计每个推理步骤的优势值，即包含该步骤对最终正确答案的贡献。实验发现，尽管足够强大的大语言模型(LLM)判断器能优于基线，但仍远未达到噪声上限。微调后的模型在错误回答上表现良好，但在正确回答上仍与上限有差距，表明推理步骤的重要性仅能从文本中部分恢复。
Legibility is Not Interpretability: Comparing Judged and Actual Importance in Chain-Of-Thought Reasoning
Reasoning traces from chain-of-thought models appear to offer a legible window into how a model arrives at its answer. A growing body of work treats them as such, using LLM judges to diagnose errors, evaluate faithfulness, and provide step-level supervision via process reward models and generative critics. These practices rely on the text of a reasoning step carrying information about its functional role. But does the text actually encode information about which reasoning steps matter? We operationalize the importance of a reasoning step as its advantage: the change in expected reward, e.g., producing the correct final answer, from including that step, estimated via Monte Carlo rollouts. Basing ground truth on these estimates, we evaluate whether LLM judges can identify high-advantage steps and find that sufficiently capable LLMs can outperform a prevalence baseline but fall well short of a noise ceiling. Fine-tuning a model as a step-level critic yields strong improvement for incorrect responses but remains distant from ceiling for correct responses, suggesting that step importance is only partially recoverable from the text of the reasoning trace. Our findings contribute to a growing body of chain-of-thought faithfulness work that cautions against treating the legibility of reasoning traces as interpretability, especially with implications for process reward modeling.