技巧精选
Tri Dao：SSM 推理缺的是缓存对了东西，不是新架构
精选理由
Tri Dao 发推聊 SSM 为什么一直吃不到投机采样和量化的红利，答案居然是缓存没选对，做推理优化的值得看看这个思路。
Tri Dao 发文指出，SSM 在渐进复杂度上占优，却一直用不上 Transformer 免费获得的投机采样和量化等推理加速，原因是缓存的对象不对。他把解决方案归为 FlashAttention、Mamba scan、paged KV 背后同一个 IO-aware 思路。他认为许多大收益并非来自新架构，而是认真对待内存 IO 的结果。
原文 · Tri Dao (FlashAttention)
SSMs had the nice asymptotics but kept missing out on the inference goodies Transformers get for free like spec decode and quantization. Turns out we were just not caching the right thing. Same IO-aware trick behind FlashAttention, Mamba scan, paged KV. A lot of the big wins weren’t new architectures, just people taking memory IO seriously.
