Sebastian Raschka 提议开源 LLM 构建模块 API
做模型实验的同学应该都体会过换一种注意力实现要多痛苦。Raschka 这条提议说到了 Megatron Core、torchtitan 这些框架都没解决掉的抽象层问题,做架构研究和消融实验的人值得看看讨论方向。
研究人员 Sebastian Raschka 发文提出,开源 LLM 生态缺少一套统一的构建模块 API。目前做消融实验时,想替换 GQA 为 MLA、把 dense FFN 换成 MoE、用 Mamba2 或 Gated DeltaNet 替代注意力层,在 PyTorch 或 C++/CUDA 下切换成本都很高,即便借助 LLM 辅助编程也很混乱。
他设想的方案可以直接架在 PyTorch 或 CUDA 之上,但需要认真工程化维护,性能要足以支撑生产环境,而不是周末快速写出来的玩具项目。他提到 torchtitan、torchtune、Megatron Core、OLMo-core 等项目已有这方面努力,但抽象层次还不够。
他也指出冷启动难题:这类 API 需要流行才能被广泛采用,又需要被广泛采用才能流行,因此可能适合由资金充足的大型实验室来推动。
The open-source LLM ecosystem could really benefit from an LLM building-block API.
We have so many LLM components across different models now. Whether you are using PyTorch or C++ and CUDA code directly, there is a major switching cost when you want to run experiments or ablation studies, say swapping GQA with MLA, dense FFNs with MoE, attention layers with Mamba2 or Gated DeltaNet, or standard residual connections with attention residuals.
Currently, it’s very messy even with LLM-assisted coding.
It could maybe be on top of PyTorch or CUDA directly, but it needs to be carefully engineered and maintained (versus quickly vibe-coded on a weekend) and efficient enough that people trust it to use in production (which is really a requirement to move beyond toy examples).
Maybe some food for thought for a big, well-funded lab :)
(There are some efforts that go into that direction, like torchtitan, torchtune, Megatron Core, OLMo-core, but they are not quite that level of abstraction, I'd say.
And, as always, there’s the chicken-and-egg problem: it needs to be popular to get widely adopted, and it needs to get widely adopted to become popular.)
本条情报归属 AITOP「技巧」核心追踪栏目。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:PyTorch、Megatron Core、Mamba2、MoE,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始线索采编自「Sebastian Raschka」(@rasbt),经 AITOP 多维评分管线结构化提炼。