Anyscale 工程师直播：用 Ray Serve 部署与优化 LLM 推理
Anyscale 的工程师要开课讲 Ray Serve 部署 LLM，从 KV cache 优化到接 Cursor、Claude Code 都覆盖，还帮你算自建 GPU 和 API 哪个更省钱。
Anyscale 工程师 Marwan 将在一场直播课中讲解 Ray Serve LLM 的部署与扩缩容，覆盖 vLLM 和 SGLang 两种推理引擎。课程内容包括 KV cache、paged attention、prefix caching 和 continuous batching 等推理优化手段，以及 KV-cache-aware routing、prefill/decode 分离和 MoE 模型的 wide expert parallelism 等大规模扩展技术。最后还会演示如何把自托管模型接入 Claude Code、Cursor、Codex 等编程助手，并对比自建 GPU、订阅制与按 token 计费三种方式的成本。
In this session, @Marwan1112, AI Engineer at Anyscale, will walk through: - Building Ray Serve applications with deployments, composition, and load-aware request routing - Deploying and scaling LLM endpoints with Ray Serve LLM on engines like @vllm_project and @sgl_project - Optimizing the inference engine with the KV cache, paged attention, prefix caching, and continuous batching - Scaling frontier models with KV-cache-aware routing, prefill/decode disaggregation, and wide expert parallelism for mixture-of-experts (MoE) models - Deploying Custom LLMs on Anyscale and Integrating Them with Agentic Platforms (Claude Code, Cursor, Codex) - Comparing costs across self-hosted GPUs, subscriptions, and token-based API billing.
Register: https://t.co/uKeb1R6hQ5
