vLLM 从 Berkeley 学生项目成长为开源推理引擎，背后的 Inferact 融资 1.5 亿美元
In 2022, Berkeley PhD student Woosuk Kwon thought he could make Meta's new open-weight model, OPT-17...
两个博士生优化推理速度的副业项目，如今跑在 50 万张 GPU 上，Google 和 Meta 都在用，还融了 1.5 亿美元。
2022 年 Berkeley 博士生 Woosuk Kwon 计划用两周优化 Meta 的 OPT-175B 推理速度，这个项目演变成 vLLM。团队借鉴操作系统的虚拟内存机制，按需分配显存，让每个 GPU 同时处理大小差异悬殊的请求。vLLM 目前有约 3,000 名贡献者，同时在约 50 万张 GPU 上运行，首个用户 LMArena 借此将 GPU 需求减半。Google、Amazon 和 Meta 都在生产环境中使用 vLLM。2026 年 1 月，核心维护者创办 Inferact 并融资 1.5 亿美元，在旧金山和新加坡招聘。
In 2022, Berkeley PhD student Woosuk Kwon thought he could make Meta's new open-weight model, OPT-17...
In 2022, Berkeley PhD student Woosuk Kwon thought he could make Meta's new open-weight model, OPT-175B, run faster in two weeks. Four years and 3,000 contributors later, that project is @vllm_project , one of the largest open-source inference engines, running on ~500,000 GPUs at any given time. He and fellow researchers Simon Mo, Kaichao You, and Roger Wang borrowed "virtual memory" from operating systems, handing out memory only as it's needed so each GPU can juggle requests of wildly different sizes. (The "v" in vLLM is for virtual.) Their first user, Berkeley neighbor @arena , cut its GPU needs in half. Today Google, Amazon, and Meta all run vLLM in production. In January, the core maintainers founded @inferact and raised $150M to give vLLM a full-time team. They're hiring across eng, marketing, and devrel in SF and Singapore: a16zjobs.substack.com/p/inferact-rai… @simon_mo_ @woosuk_k @KaichaoYou @rogerw0108 Your browser does not support the video tag. 🔗 View on Twitter 💬 0 🔄 0 ❤️ 0 ⚡
