论文

Atomic MathReasoner:用原子推理技能组合提升 LLM 数学能力

Atomic MathReasoner: Instance-Level Composition of Atomic Reasoning Skills for Mathematical LLMs

精选理由

把数学推理拆成 LoRA 专家库再按题路由,思路和传统 MoE 不太一样,用的是无梯度优化加 NLL 打分。7B 模型逻辑推理提了 7 个点,做模型合并或路由的人可以看看方法细节。

一篇 arXiv 论文提出 Atomic MathReasoner(AMR),一个实例级框架,把数学推理拆解为原子技能再按题目组合,而非作为整体能力训练。AMR 先训练一组能力专属的 LoRA 专家,然后在留出的验证子集上做无梯度优化,发现多个低损失组合原型;再用参考解答的 token 级负对数似然评估每个原型的偏好,通过置信度边际过滤模糊分配,最后把伪标签蒸馏进一个轻量路由器。推理时路由器为每道题选择合适的组合原型,避免昂贵的在线优化。

正文

在 AtomicMath 基准上,Qwen2.5-Math-Instruct-7B 的 Level 1 平均分项准确率从 64.4% 提到 68.1%,Level 2 从 59.3% 提到 63.7%,平均逻辑推理从 39.4% 提到 46.8%,通用型与数学专用型 LLM 上均有一致提升。对做模型组合与 MoE 路由的开发者来说,这套“专家库 + 无梯度原型发现 + 蒸馏路由”的流程值得参考。

原文 · arXiv cs.LG

Atomic MathReasoner: Instance-Level Composition of Atomic Reasoning Skills for Mathematical LLMs

Most existing approaches enhance mathematical reasoning in large language models (LLMs) as a monolithic capability, despite the fact that different mathematical problems require different reasoning skills. Recent work has explored a more fine-grained perspective by decomposing mathematical reasoning into atomic skills and developing specialized modules for individual capabilities. However, how to effectively utilize these atomic abilities remains underexplored. To address this challenge, we propose Atomic MathReasoner (AMR), an instance-level framework for composing atomic reasoning skills. AMR first trains a library of capability-specific LoRA experts and performs gradient-free optimization over held-out validation subsets to discover a diverse set of low-loss composition prototypes. It evaluates the preference of each prototype using the token-level negative log-likelihood of reference solutions, filters ambiguous assignments through a confidence margin, and distills the resulting pseudo-labels into a lightweight router. During inference, the router selects an appropriate composition prototype for each problem, enabling problem-specific utilization of atomic skills without costly online optimization. Experiments on the AtomicMath benchmark across general and mathematics-specialized LLMs demonstrate consistent improvements. On Qwen2.5-Math-Instruct-7B, AMR increases average field accuracy from 64.4% to 68.1% at Level 1 and from 59.3% to 63.7% at Level 2, while improving average logical reasoning performance from 39.4% to 46.8%. These results demonstrate that effectively composing specialized atomic skills provides a promising approach to strengthening mathematical reasoning in LLMs.

产业坐标与信源核验

本条情报归属 AITOP「论文」核心栏目,重点聚焦中国 AI 产业与国产大模型生态。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:Atomic MathReasoner、Qwen2.5-Math-Instruct-7B、LoRA、数学推理,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「arXiv cs.LG」(@Jiashu Li, Tianyu Gao, Qinglong Liu, Jinping Wang, Ye Wei, Zhiqiang Gao),经 AITOP 多维评分管线结构化提炼。