新研究提出决策审计方法：模型执行成功不代表产品决策正确
The Delegation Blind Spot: Auditing Product Decisions from Agent Choices
这篇论文用 4,800 次请求证明模型跑得准不等于选对了产品方向，还给出不用调用模型的提取器，做 Agent 评估的可以看看。
arXiv 论文提出一种针对 Agent 决策的审计方法，将声明的观察通道与产品价值对比映射为兼容区间和见证人群。冻结实验向两个固定版本的模型快照发出 4,800 次请求，36 个保守主区间全部未获解决。后续 2,400 次调用仅解决每模型九项对比中的三项，而确定性提取器在不调用模型的情况下解决了九项中的七项。论文还提供带来源标签的决策回执和离线查看器，建议在收集更多遥测数据前先诊断决策为何无法确定。
The Delegation Blind Spot: Auditing Product Decisions from Agent Choices
Successful agent execution need not identify which future product improvement its user would value. We present a decision-specific audit that maps a declared observation channel and product-value contrast to compatible intervals and witness populations. Its foundations are established identification and decision theory; the contribution is an executable measurement workflow and a controlled study of its limits. A frozen experiment makes 4,800 requests to two pinned model snapshots on shared synthetic tasks. All 36 conservative primary intervals remain unresolved despite different execution accuracy. An exploratory 2,400-call follow-up records supplied preferences and resolves three of nine comparisons per model. A deterministic extractor resolves seven of nine without model calls or calibration observations, exposing unnecessary uncertainty introduced by model-generated reports. A further 14,400 controlled multinomial simulations distinguish structural ambiguity from weak identification and finite calibration precision. We propose a source-labeled decision receipt and provide an offline viewer for inspecting the audit. These results motivate preserving decision-relevant structured input and diagnosing why a decision is unresolved before collecting more telemetry. The study contains no human participants or real customer outcomes. Full proofs, raw model provenance, controlled experiments, and reproducible analyses accompany the report.
