World 模型排序策略不必完全准确：相对误差界与审计极限
World Models Need Not Be Right to Rank Policies Right: Relative Bounds and Audit Limits
这篇 arXiv 论文提出 RAVEL 方法，用 world 模型低成本给 RL 策略排序，淘汰阶段再用少量真实审计兜底，还量化了模型排序可靠到什么程度、什么时候会失灵。
论文研究用学习型 world 模型评估 RL 策略排序的问题，提出精确成对仿真恒等式和相对误差界，区分共模 Bellman 残差与策略特定误差。作者提出 RAVEL，一种淘汰环节使用真实环境结果的选择性策略锦标赛方法。在 DMC cheetah-run 和 walker-walk 上测试每任务 45 个 SAC、TD3、PPO 策略，rollout 层级 500 时 Kendall tau 达 0.962 和 0.800。实验同时显示 8 次审计下同时区间覆盖率 99.98% 和 99.75%，但仅能认证 13.2% 和 0% 的决策，且低预算下 RAVEL 改善 walker-walk regret，高预算下真实环境 successive halving 更优。
World Models Need Not Be Right to Rank Policies Right: Relative Bounds and Audit Limits
Selecting a reinforcement-learning (RL) policy usually requires additional environment interaction, while learned world models offer cheap but potentially biased evaluations. This raises two distinct questions: can shared model errors preserve pairwise rankings despite poor absolute values, and do they make a small number of real audits more informative? We derive an exact pairwise simulation identity and a relative bound separating common-mode Bellman residuals from policy-specific error. The analysis motivates RAVEL (Relative Auditing of Value Errors in Learned models), a selective policy tournament whose eliminations use held-out real outcomes. We test 45 SAC, TD3, and PPO policies per task in DMC cheetah-run and walker-walk. At rollout horizon 500, model rankings remain strong (Kendall tau = 0.962 and 0.800), and same-run differential-error ratios are 0.674 and 0.106. Yet mean paired audit-error correlations are only 0.007 and 0.090, and pairing changes standard deviation by -2.5 percent and +1.9 percent. Simultaneous intervals attain 99.98 percent and 99.75 percent coverage, but certify only 13.2 percent and 0 percent of decisions at eight audits. RAVEL improves low-budget walker-walk regret, whereas real-only successive halving is better at the largest budget, rejecting the precommitted twofold efficiency target. Frozen-model optimization lowers real return without increasing predicted return on average. Thus ranking fidelity, useful audit covariance, and upward model exploitation are distinct properties, separated in part by closed-loop trajectory divergence.