层级贝叶斯分析显示：推理模型规模扩大能力递减，效率几乎不变
Scaling of Capability and Efficiency at Inference Time in Large Reasoning Models
这项研究拿 DeepSeek-R1-Distill 全系列做实验，结论挺扎心：模型变大只是解题能力强了一点，推理消耗的 token 一点没省，想靠堆规模提效率可能走不通。
论文用层级贝叶斯模型评估了 DeepSeek-R1-Distill 系列在 4 类算术与算法推理问题上的表现。固定模型规模下，解题正确率随题目规模（难度代理指标）呈近似指数衰减，衰减尺度随模型规模亚线性增长，说明能力提升存在递减效应。输出 token 长度随题目规模呈幂律增长，但幂律参数不随模型规模系统性变化，表明更大模型并未变得更高效。作者据此指出单纯堆算力扩展可能同时面临能力收益递减与效率停滞两个限制。
Scaling of Capability and Efficiency at Inference Time in Large Reasoning Models
Capability and efficiency are two key dimensions of reasoning in large language models (LLMs). Capability refers to the ability to solve a given problem correctly, whereas efficiency refers to the ability to do so with limited resources. When LLMs use Chain-of-Thought (CoT) reasoning to solve problems of controlled hardness, both the number of problems solved correctly and the number of tokens required to reach a correct answer depend on problem hardness and model size. However, how these factors jointly shape capability and efficiency remains poorly understood. Here, we use hierarchical Bayesian models to evaluate the capability and efficiency of LLMs from the DeepSeek-R1-Distill model family across four classes of arithmetic and algorithmic reasoning problems. At a fixed model size, the probability of correctly solving an instance decays approximately exponentially with instance size, our proxy for problem hardness. The decay scale grows sublinearly with model size, indicating that larger models are more capable, but that capability gains diminish with scale. Output length grows as a power law with instance size, which serves as a proxy for difficulty. However, the parameters of this power law do not vary systematically with model size, suggesting that larger models do not become more efficient. Together, these findings reveal potential limitations of naive scaling as a strategy for developing more capable AI systems: capability improves with diminishing returns, while efficiency shows little to no improvement.
