Particle解码免费估计序列级Rényi熵，自适应推理降低算力开销
Free Lunch Uncertainty: Sequence-Level Rényi Entropy from Particle Decoding for Adaptive LLM Reasoning
做推理部署的可以看看：SMC 解码顺手就能算出序列级熵，不用额外跑补全。论文在 MATH500 上验证过，两遍策略用少 34.2% 的粒子就能保住 64 粒子的准确率，算力紧张的团队直接受益。
一篇新论文提出从 Sequential Monte Carlo（SMC）power sampling 解码中零成本读出序列级 Rényi 熵（SRE）：由于 SMC 本身已估计 power 分布的归一化常数，序列级熵可在任意粒子预算下免费获得，无需像 self-consistency 那样额外生成大量补全。作者用 Rényi 链式法则将读数扩展为一族逐步熵，并证明阶数大于 1 的成员与每步最可能的 token 绑定（因此与 log-probability 相关），而 1/2 阶成员累积每步搜索努力的对数。
实验显示该 1/2 阶信号在 13 个设置中的 10 个里是最好的零成本选择性预测信号；在 MATH500 上其 AUROC 比 log-probability 高 0.022 到 0.044，且在全部四个模型和各长度四分位内保持一致，held-out 设置下也选出同一阶数。
在实际应用上，一个 8 粒子探针可识别哪些问题需要更多粒子，配合两遍策略在 MATH500 上用少 34.2% 的粒子匹配 64 粒子的准确率（三个种子平均）。这意味着已在使用 SMC/power sampling 的团队可以免费获得不确定性估计并显著削减推理算力，而 token 级熵和 log-probability 等现有零成本信号只覆盖局部信息。
Free Lunch Uncertainty: Sequence-Level Rényi Entropy from Particle Decoding for Adaptive LLM Reasoning
Quantifying uncertainty over an entire LLM completion is critical for selective prediction and adaptive compute. Zero-cost signals such as mean token entropy and log-probability are local to tokens or to one completion, and self-consistency needs many extra completions. We show that Sequential Monte Carlo (SMC) decoding for power sampling already estimates the normalizing constant of the power distribution, which gives the sequence-level R'enyi entropy (SRE) of the model's completion distribution at zero marginal cost, with a guarantee at every particle budget. A R'enyi chain rule extends this readout to a family of stepwise entropies of any order. We prove that orders above one are tied to the most likely token of each step, which explains why they track log-probability, while the order-1/2 member accumulates the log of per-step search effort. This member is the best zero-cost selective-prediction signal in 10 of 13 settings. On MATH500 its AUROC is 0.022 to 0.044 above log-probability with all four models and within length quartiles, and the same order is selected when each setting is held out. An 8-particle probe identifies the problems that benefit from more particles, and a two-pass policy matches 64-particle accuracy on MATH500 with 34.2% fewer particles over three seeds.
本条情报归属 AITOP「论文」核心栏目，重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中，该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：Rényi熵、Sequential Monte Carlo、MATH500、不确定性量化，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明：本文原始线索采编自「arXiv cs.LG」（@Erfan Baghaei Potraghloo, Milad Heidari Sorkhehdizaj, Shahab Rahimirad, Aliasghar Pourghani, Seyedarmin Azizi, Souvik Kundu, Abolfazl Hashemi, Joseph Campbell, Vijay Gupta），经 AITOP 多维评分管线结构化提炼。