Jev-Mobile:低频 VLM 规划+高频轻量执行,加速移动端 GUI 智能体
Jev-Mobile: Jev as an Executor for Mobile GUI Agents
一篇让手机自动化代理又快又省钱的论文:VLM 只做规划,小模型 Jev 负责点按操作,API 开销直接砍掉七成多。
Jev-Mobile 把移动端 GUI 智能体拆成两层:VLM 低频规划局部目标,轻量决策模型 Jev 高频基于 accessibility tree 选择具体动作。在 AndroidWorld 全套任务上,Jev-Mobile 达到 79% 成功率,略高于 SeeAct-V 的 78%,低于 Step-wise VLM 基线的 84%。但在成功的轨迹中,相比 Step-wise VLM,平均端到端执行时间缩短 32.7%,模型 API 成本降低 73.4%。
Jev-Mobile: Jev as an Executor for Mobile GUI Agents
Vision-language models (VLMs) have become a common foundation for autonomous mobile GUI agents, but most existing systems rely on the VLM for both planning and action grounding at nearly every interaction step, leading to substantial latency and model-serving cost. We introduce Jev-Mobile, which shifts this paradigm to low-frequency VLM planning and high-frequency lightweight execution: the VLM specifies local goals, the accessibility tree defines a structured executable action space, and Jev, a fast typed decision model, repeatedly selects actions within this space. This design allows multiple GUI actions to be executed under a single VLM decision, reducing expensive VLM inference while preserving adaptive interaction. On the full AndroidWorld task suite, Jev-Mobile achieves 79% task success, compared with 78% for SeeAct-V and 84% for a Step-wise VLM baseline. Among successful trajectories, it reduces mean end-to-end execution time by 32.7% and mean model API cost by 73.4% relative to Step-wise VLM. These results show that decoupling high-level VLM reasoning from low-level action execution can substantially improve mobile GUI agent efficiency while maintaining competitive task performance.