KVCache Meetup Korea：CXL 内存池化与 LLM 推理优化实践分享
韩国开发者聚会的技术干货：SK hynix 和 FuriosaAI 等团队拆解 LLM 推理 serving 的内存和调度优化，做部署的可以看看。
在 PyTorchKR 与 XCENA 联合主办的 KVCache Meetup Korea 上，XCENA、SK hynix、FuriosaAI 和 LMCache 四家公司的工程师分享了 LLM serving 的架构权衡。议题包括基于 Maru 的 CXL 内存池化、prefill 与 decode 阶段 GPU 算力和 HBM 带宽的 roofline 分析、RNGD 芯片上的 batch 调度，以及 LMCache MP Mode 的进程隔离缓存服务。演讲者强调，真实优化需要从加速器、内存架构到运行时引擎的每一层协同。
At the recent KVCache Meetup Korea, co-hosted by PyTorchKR and XCENA, speakers from XCENA, SK hynix, FuriosaAI, and LMCache dove deep into the architectural trade-offs powering large language model serving.
Talks spanned CXL memory pooling with Maru, roofline analysis of GPU compute and HBM bandwidth during prefill and decode, batch scheduling on RNGD, and process-isolated cache serving with LMCache MP Mode.
Real-world optimization requires collaboration across every tier of the stack, from accelerators and memory architectures to runtime engines. Thank you to PyTorch Foundation Ambassador Junghwan Park, co-host XCENA, and sponsor @tensormesh for bringing this local developer community together.
Catch the full event recap here: https://t.co/FgDtr9XgFO
- CNA: Business11-02 04:19原文
