Hibana：面向统一内存设备的本地智能体推理引擎
Hibana: Fast Agentic Inference on Unified-Memory Devices
想在本地跑大模型智能体的开发者可以看这篇。它专门针对 DGX Spark 这类统一内存设备优化，跑 125B 的 Qwen、Nemotron 比 vLLM、llama.cpp 快 3 到 4 倍，还能保存智能体状态随时恢复，代码也开源了。
Hibana 是一个专为统一内存设备设计的推理引擎，在单台 DGX Spark 上可服务最高 125B 参数的 Qwen 和 Nemotron 模型。相比 vLLM、NVIDIA Flare、llama.cpp 和 TensorFold，它每小时完成最多 3.4 倍的智能体任务，解码吞吐量最高提升 3.9 倍。现有主流引擎如 vLLM 和 SGLang 面向配备独立主存的数据中心 GPU，本地引擎则未能充分利用统一内存设备的内存层级。
技术上，Hibana 围绕统一内存系统级芯片的共享内存容量和有限带宽做了三项协同优化：通过模型手术（model surgery）修改训练后的模型架构以缩减内存占用和数据搬运；统一内存管理保留每个智能体跨轮次的状态，使长时运行的智能体可以即时恢复而无需重算；自适应投机解码在每步生成多个 token。代码已在 GitHub 的 sharar-project/hibana 仓库开源。
Hibana: Fast Agentic Inference on Unified-Memory Devices
Local AI promises capable personal assistants that run on users' own devices, keeping data and computation under their control. Open-weight models that fit on local devices can now handle practical tasks such as coding. Yet current inference engines fall short. Popular engines such as vLLM and SGLang target datacenter GPUs with dedicated host memory, while local engines underuse the memory hierarchy of unified-memory devices. We present Hibana, an inference engine for unified-memory devices that co-optimizes model representation, memory management, and decoding strategy. Model surgery, the process of editing a trained model's architecture to suit its hardware, changes how the model computes, shrinking its memory footprint and data movement. Unified-memory management preserves each agent's state across turns, so long-running agents can resume instantly instead of recomputing their state. Adaptive speculation generates several tokens per step. Hibana codesigns these features around the shared memory capacity and limited bandwidth of unified-memory systems-on-chip to support local multi-agent workloads. On a single DGX Spark serving Qwen and Nemotron models of up to 125B parameters, Hibana completes up to 3.4$\times$ more agentic tasks per hour and delivers up to 3.9$\times$ higher decoding throughput than other established frameworks such as vLLM, NVIDIA Flare, llama.cpp, and TensorFold. Code is available at https://github.com/sharar-project/hibana.
本条情报归属 AITOP「模型」核心栏目，重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中，该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」「中美大模型技术代差与商业化对照」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：Hibana、vLLM、llama.cpp、DGX Spark，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明：本文原始线索采编自「arXiv cs.AI」（@Yahya Emara, Chi-Chih Chang, Anthony Fei, Ahmed F. AbouElhamayed, Chong-Yan Chen, Hung-Yueh Chiang, Kai-Chiang Wu, Mohamed S. Abdelfattah），经 AITOP 多维评分管线结构化提炼。截至目前，全网已有 11 家独立信源对该事件进行交叉印证与跟进报道（含 IT之家、vLLM、techcrunch 等），属于近期高关注度行业信号。
