斯坦福团队新论文提出Web Agent世界模型新训练方法,在WebArena基准上表现更好。
研究人员提出了一种名为predicted-state matching的新型训练目标,用于Web Agent的世界模型训练。该方法使用从WebArena Go-Browse轨迹中生成的分支式数据集,使模型能够区分真实结果状态与其他替代动作导致的状态。实验表明,该方法在预测状态匹配基准上优于传统监督式预测模型,在WebPRMBench上提升了PRM式动作排名,并在WebArena-Lite上提高了端到端任务成功率。
Discriminative World Models for Web Agents
Recent web agents use world models for test-time action selection by sampling candidate actions, predicting the resulting web states, and ranking them with a ranker model or a Process Reward Model (PRM). These world models are typically trained via supervised next-state prediction to generate fixed representations like HTML or AXTree snapshots. However, this objective is misaligned with the downstream ranker, which relies on predicted states being discriminative across candidates to accurately score them. To address this, we introduce predicted-state matching, a training objective where the predicted representation must distinguish the true resulting state from those reached by alternative actions. We train these models using a branching web-agent dataset derived from WebArena Go-Browse trajectories, where every decision point contains multiple alternative actions and their resulting states. Experiments on our held-out predicted-state matching benchmark show that our approach outperforms world models trained with supervised next-state prediction. We further show that our approach improves PRM-style action ranking on WebPRMBench compared with action-only PRMs and PRMs augmented with supervised-next-state world models. Finally, on WebArena-Lite, using our world model for test-time action selection improves end-to-end task success. Our project page is available at: https://dhruvpendharkar.github.io/dwm/.