用 Jev 决策模型替代大语言模型做边缘服务编排，延迟降15.9%-26.5%
Replacing Large Language Models with Jev Decision Models for Low-Latency Edge Service Orchestration
有人拿 Jev 决策模型替掉 DeepSeek 和 Qwen 跑边缘编排，延迟最多降26.5%，API 费用省近七成。
论文将 Jev 的决策导向 API 接入边缘服务编排，通过提取四个有界意图字段并复用共享验证器、准入策略和调度器，取代自然语言模型的决策环节。对比基线包括结构化输出的 DeepSeek 部署和自托管 Qwen 的真实两节点 OCR 服务。三个连续测量块中，Jev 将客户端决策中位延迟降低 15.9%-26.5%。八组配对 OCR 条件下，Jev 在七组中与 DeepSeek 的正确按时完成数持平、一组反超；无缓存时端到端中位延迟低 11.1%-25.3%，每次正确完成的 API 费用低 69.0%-70.6%。重复请求缓存基本消除延迟差异，测试路径中的主要节省机会在于新请求的解释环节。
Replacing Large Language Models with Jev Decision Models for Low-Latency Edge Service Orchestration
Natural-language service requests can require a language-model decision before execution starts, consuming part of the request's latency budget. We integrate Jev's decision-oriented application programming interface (API) into edge service orchestration to reduce this overhead while retaining service completion. The integration extracts four bounded intent fields and applies a shared validator, admission policy, and scheduler, accounting for decision waiting throughout the request timeline. We compare Jev with a short, structured-output DeepSeek deployment using live API measurements followed by modeled execution, and then a real two-node optical character recognition (OCR) service with self-hosted Qwen and rule-based references. Across three consecutive measurement blocks, Jev reduces median client decision latency by 15.9-26.5%. In eight paired OCR conditions, Jev matches DeepSeek's correct, on-time completion count in seven and exceeds it in one. Without caching, median end-to-end latency on requests completed correctly by both systems is 11.1-25.3% lower; API fees per correct completion are 69.0-70.6% lower. Repeated-request caching largely removes the latency difference. The results demonstrate that decision-model substitution can lower both response latency and API fees in the tested service path, and identify fresh interpretation as the main opportunity for latency savings.
