ARK：为分离式 LLM 推理中的 KV Cache 传输解决路由冲突
ARK: Avoiding Routing Collisions for KV Cache Transfer in Disaggregated LLM Inference
KV Cache 传输是网络里的巨流，ECMP 哈希一撞就堵。ARK 只调源端口就能避开，FCT 降了 27.3%，做推理基础设施的可以看看。
论文提出 ARK，一种针对分离式 LLM 推理中 KV Cache 大流传输的分布式路径预留机制。分离式推理将 prefill 和 decode 阶段部署在不同 GPU 池，传输持续数百毫秒到数秒，在 ECMP 下易发生哈希冲突，导致部分链路过载。ARK 通过协调发送方选择源端口，让并发流映射到不同 spine，无需修改交换机或接收端重排包。RDMA 包级仿真显示，ARK 在中等负载下将平均 FCT 和 P95 FCT 分别降低 27.3% 和 34.0%，平均 TTFT 降低 12.9%。
ARK: Avoiding Routing Collisions for KV Cache Transfer in Disaggregated LLM Inference
Disaggregated LLM inference separates the prefill and decode phases across GPU pools, generating massive KV-cache transfers. Because these transfers last hundreds of milliseconds to seconds, they behave as mega elephant flows that dominate link bandwidth utilization. In this regime, stateless ECMP can perform poorly: hash collisions may overload one spine link while leaving others idle, stretching transfer times by seconds. Yet this same persistence makes coordination practical. Since these flows are long-lived, even lightweight one-to-all coordination can be amortized over their lifetime. We present ARK, a distributed elephant-flow path reservation mechanism. ARK coordinates senders to choose source ports whose hashes map concurrent flows onto distinct spines, without requiring switch changes or receiver-side packet reordering. Packet-level RDMA simulations show that ARK reduces mean and P95 FCT by up to 27.3% and 34.0% under moderate load, and further reduces mean TTFT by up to 12.9%.
