论文

线性注意力模型状态分页上 FPGA:实测仅占解码开销 11.2%

The State Is Not the Bottleneck: A Paged-State Gated-DeltaNet Tile on a Tri-Mode Multicast Mesh, Bit-Exact from RTL to Silicon

精选理由

搞边缘部署的值得看这篇:团队在 Agilex 5 上实测了 Gated DeltaNet 的状态分页开销,结果发现分页只占 11.2%,权重搬运才是真瓶颈。做 FPGA 推理或线性注意力硬件化的人能直接参考这些数字。

一篇新论文验证了线性注意力语言模型在边缘 FPGA 上的一个关键问题:固定大小的循环状态需要分页到片外内存时,分页是否会成为解码瓶颈。研究团队在 Agilex 5 FPGA 上构建硬件 tile,运行一个 3.69 亿参数的混合 Gated DeltaNet 模型的完整单层计算。该模型状态需要 6.2 MiB,而芯片片上 block memory 只有 0.87 MiB,每解码一个 token 产生 26.0 MB 的内存流量。

正文

实测结果显示,分页最多只占解码总开销的 11.2%,权重矩阵-向量乘积反而占约 62% 的时间,状态阶段(含分页)仅占 13%。板载 75 MHz 频率下(合成权重)单 token 该层耗时 11.15 ms。验证方面,仿真中状态逻辑与定点参考在全部 18 个循环层上逐位一致;板级测试中六个构建版本通过了 30 次 64 token 和 2 次 1,024 token 的全部测试,定点算术误差也对照浮点做了量化。

结论对边缘部署有直接参考价值:在这类模型和内存接口下,限制解码性能的是权重传输而非循环状态,状态分页在边缘 FPGA 上是可行方案。

原文 · arXiv cs.LG

The State Is Not the Bottleneck: A Paged-State Gated-DeltaNet Tile on a Tri-Mode Multicast Mesh, Bit-Exact from RTL to Silicon

Linear-attention language models replace a growing key-value cache with a recurrent state of fixed size. On an edge FPGA, this state can exceed the on-chip memory and must be paged to external memory. This paper asks whether such paging dominates the cost of decoding (generating tokens). We built a hardware tile that computes one complete layer of a custom 369-million-parameter hybrid Gated DeltaNet model on an Agilex 5 FPGA and pages its recurrent state. The model state needs 6.2 MiB, whereas the device has 0.87 MiB of block memory. Paging adds 26.0 MB of memory traffic per decoded token, at most 11.2 percent of the calculated total. On the board at 75 MHz, with synthetic weights, one token takes 11.15 ms in this layer. The matrix-vector products with the weights take about 62 percent of this time and the state phase, including the paging, 13 percent. The tile is one node of a multicast mesh of tri-mode processing elements. We measure the error of the fixed-point arithmetic against floating point and check that the hardware agrees bit for bit with a fixed-point reference. In simulation, the state logic agrees with the reference on all 18 recurrent layers. On the board, six builds pass all 30 runs of 64 tokens and two runs of 1,024 tokens. For this model and memory interface, the delivery of the weights limits the decoding performance, and the recurrent state does not. State paging is therefore practical for such a model on an edge FPGA.

产业坐标与信源核验

本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:Gated DeltaNet、线性注意力、FPGA、边缘推理,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「arXiv cs.LG」(@Cong Xu, Ravi Sankar),经 AITOP 多维评分管线结构化提炼。