EngineAgent：用智能体自动化端侧 LLM 推理引擎的模型适配
EngineAgent: Agentizing On-Device LLM Inference Engines with Inference Stack Hierarchy Awareness and Hardware Abilities
端侧适配新模型要在转换、图构建、算子、后端好几层同时改，普通编码 agent 搞不定。这篇论文的 builder+auditor 双角色设计把成功率从 33.3% 拉到 75%，做端侧部署的工程师值得细看实现思路。
论文提出 EngineAgent，一个自动化端侧 LLM 模型适配的智能体系统，在 12 个任务（3 个移动端 LLM、4 组引擎-后端组合，覆盖 CPU/GPU/NPU）上实现 75% 的端到端成功率，而最强编码智能体基线仅 33.3%。
核心机制是把可复用的引擎与硬件知识从模型特定实现中分离：适配前先构建感知引擎抽象层级与跨层依赖的引擎索引，并用已支持模型预验证构建、执行、调试流程；适配新模型时由 builder 协调转换、图构建、算子、硬件后端等跨层改动，auditor 负责完整性检查并通过设备端验证驱动迭代调试。
成功标准要求编译通过、设备上可执行且输出 logit 与参考实现一致，同时平均单任务耗时和模型 API 成本低于同底座模型的基线，对做端侧部署的工程团队有直接参考价值。
EngineAgent: Agentizing On-Device LLM Inference Engines with Inference Stack Hierarchy Awareness and Hardware Abilities
Rapidly evolving LLM architectures and heterogeneous mobile hardware make supporting new models in on-device inference engines a recurring and costly system challenge. Each model may require coordinated changes across conversion, graph construction, operators, and hardware backends. Existing coding agents often miss these cross-layer dependencies and struggle to diagnose failures across host--device execution paths. We present EngineAgent, an agentic system for automating on-device LLM model enablement. Its key insight is to separate reusable engine and hardware knowledge from model-specific implementation. Before enablement, EngineAgent constructs a hierarchy-aware engine index that captures engine abstraction layers and cross-layer dependencies, and pre-validates backend-specific build, execution, and debugging workflows using already supported models. For each new model, a builder uses this knowledge to coordinate cross-layer changes, while an auditor checks implementation completeness and drives iterative debugging through device-side validation. Across 12 tasks spanning three mobile LLMs and four engine--backend pairs on CPUs, GPUs, and NPUs, EngineAgent achieves 75% end-to-end success with either of two underlying LLMs, versus 33.3% for the strongest coding-agent baseline. Success requires compilation, on-device execution, and output-logit agreement with reference implementations. EngineAgent also reduces average per-task time and model-API cost relative to baselines using the same underlying LLMs.
本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中，该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：EngineAgent、端侧推理、智能体、LLM 推理引擎，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明：本文原始线索采编自「arXiv cs.AI」（@Zhengxiang Huang, Yujie Sun, Shengheng Chen, Zhaode Wang, Chaoyue Niu, Zeyu Zhao, Chengfei Lv, Fan Wu, Guihai Chen），经 AITOP 多维评分管线结构化提炼。