论文

用条件分布检验定位智能体与玩家行为的分歧

When Agents Play Differently: Localizing Human--Agent Behavioral Divergence with Conditional Distributional Testing

精选理由

搞游戏 bot 测试或 agent 评测的可以看看这篇:它不比平均分,而是按上下文分组定位 bot 在哪类模式里玩得不像人,代码也开源了。比单看胜率或聚合分数的常规评测精细得多,做自动化 playtesting 的团队用得上。

一篇 arXiv 论文提出分层检验框架,在上下文分格内计算非参数差异统计量,并自动发现智能体行为偏离玩家的子群规则。

正文

核心发现是:会话级指标以输赢结果为主,其信号量不超过均值比较;而走子效率等过程级指标才能揭示真实的行为差异。

作者将该方法应用于一款商业三消游戏的 playtesting bot,结果显示该 bot 在部分游戏模式下与人类玩家存在显著分歧,在另一些模式下则未检测到分歧,这类模式级的差异在聚合指标下不可见。实现代码已开源,便于复现与落地。

原文 · arXiv cs.AI

When Agents Play Differently: Localizing Human--Agent Behavioral Divergence with Conditional Distributional Testing

Evaluating whether artificial agents behave like humans requires comparing not just average outcomes but entire behavioral distributions across diverse contexts. We propose a stratified framework that computes nonparametric discrepancy statistics within contextual cells, and discovers subgroup rules identifying where agent behavior departs from human play. We show that session-level metrics dominated by win/loss outcomes offer little signal beyond mean comparisons, whereas process-level metrics such as move efficiency reveal genuine behavioral differences. Applied to a commercial match-3 game, it uncovers patterns invisible to aggregate metrics: a playtesting bot is diverging substantially from humans in certain game modes, while showing no detectable divergence in others. We release our implementation as open source to support reproducibility and adoption.

产业坐标与信源核验

本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:智能体、Agent评测、行为分布检验、游戏测试,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「arXiv cs.AI」(@Oleg Smirnov, John Pertoft, Sara Karimi),经 AITOP 多维评分管线结构化提炼。