研究测试十个 LLM 智能体在公共服务场景中的自主决策行为
Read and Ignore: What Self-Directed LLM Agents Do in Public Service Scenarios
政府机构和医院学校采购 AI 智能体前值得看看这项测试。它用 2,400 条轨迹量化了十个模型在指令模糊时到底会读档案还是自作主张，还给出了可在自家数据上直接跑的规则化评估方法，代码全部开源。
arXiv 上一项新研究构建了四个经专家验证的公共服务场景，并采集了 2,400 条来自十个 LLM 的运行轨迹，对比 1,000 次无上下文沙盒运行，考察智能体在指令不明确时的自主行为。
结果显示，开放式任务下智能体在 96.9% 的运行中会读取档案记录，但只有 18.7% 会产出工件或自选任务；给定明确任务后这一比例升至 86.0%。领域上下文并未改变产出比例（18.7% 对 16.0%），各模型产出能力的排名在不同场景间基本保持一致（Spearman 0.79），但模型是否读取记录的排名则不一致（0.19）。上下文能把每轮运行中的边界试探回合减少 60%，同时部分生成的工件与档案记录内容相矛盾。
研究团队提供了一套无需标注数据的规则化分类测试，公共服务机构可在采购前用自己的档案记录自行运行测试，相关场景、代码和轨迹数据均已开源发布。
Read and Ignore: What Self-Directed LLM Agents Do in Public Service Scenarios
Underspecified instructions leave LLM agents to decide what to do. In public services, staff write these instructions, but the agent's decisions affect residents, patients, and students. We build four expert-validated public-service scenarios with records and decision levers that can be checked in the files, collect $2{,}400$ trajectories from ten LLMs under four prompt conditions, and compare them with $1{,}000$ runs in a context-free sandbox. Agents read the records in $96.9\%$ of open-ended runs and produce an artifact or a self-chosen task in $18.7\%$; with an explicit task, $86.0\%$ produce. Domain context does not change how many runs produce ($18.7\%$ vs.\ $16.0\%$), and the production ranking of the models largely carries over (Spearman $0.79$), while the ranking by whether a model reads the records does not ($0.19$). Context cuts boundary-probing turns per run by $60\%$, and some artifacts contradict the records. We contribute a test that a public-service institution can run on its own records before procurement; its rule-based taxonomy needs no labeled data, and scenarios, code, and trajectories are released.
本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中，该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：LLM、智能体、AI安全、公共服务，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明：本文原始线索采编自「arXiv cs.AI」（@Diana Kozachek, Kinjalk Srivastava），经 AITOP 多维评分管线结构化提炼。
