Anyscale 网络研讨会：用 Ray Serve 构建可扩展的分布式推理系统
Anyscale 要开讲 Ray Serve 推理系统设计了，从 KV cache 优化到 MoE 扩展都讲，还教你算自建 GPU 和 API 计费哪个便宜。
Anyscale 宣布举办一场关于生产级分布式推理系统设计的网络研讨会，核心围绕 Ray Serve 展开。内容包括用 vLLM 和 SGLang 等引擎部署 LLM 端点，以及 KV cache、paged attention、continuous batching 等推理引擎优化手段。研讨会还会讲解 KV-cache-aware 路由、prefill/decode 分离和 MoE 模型的 wide expert parallelism 等规模化技术。另外涉及在 Anyscale 上部署自定义 LLM 并接入 Claude Code、Cursor、Codex 等编程工具，以及对比自建 GPU、订阅制与 token 计费三种方案的成本。
Our upcoming webinar will cover how to design scalable distributed inference systems for production AI applications, from classic ML models to large language models. Learn how Ray Serve delivers high-throughput, low-latency serving, and how to evaluate the performance tradeoffs in autoscaling, batching, routing, and inference-engine optimization. Topics include: - Building Ray Serve applications with deployments, composition, and load-aware request routing - Deploying and scaling LLM endpoints with Ray Serve LLM on engines like @vllm_project and @sgl_project - Optimizing the inference engine with the KV cache, paged attention, prefix caching, and continuous batching - Scaling frontier models with KV-cache-aware routing, prefill/decode disaggregation, and wide expert parallelism for mixture-of-experts (MoE) models - Deploying Custom LLMs on Anyscale and Integrating Them with Agentic Platforms (Claude Code, Cursor, Codex) - Comparing costs across self-hosted GPUs, subscriptions, and token-based API billing.
Register: https://t.co/NIFhcVRYlS
