arXiv 论文：LLM 能否识破造假的量化回测？
Can LLMs Catch a Rigged Backtest? A Clean-Control Calibration Benchmark
量化圈的朋友看过来，这论文测了 LLM 审回测靠不靠谱，DeepSeek 召回满分但一不留神就把好策略全打假，方法设计挺有意思。
arXiv 论文构建了一个含 96 组配对样本的回测审计基准，每个有缺陷的回测都配有仅改动一处方法细节的干净对照策略。
在四个文本端点共 1440 次缓存审计中，DeepSeek 审计器对闭式与 clean-aware 代码缺陷的召回率达 100.0%，但开放提示词会将 93.
8% 的干净代码误判为有缺陷。加入 clean-aware 警告后，DeepSeek 代码误报率从 20.
8%（95% CI 11.7–34.
3）降至 0.0%，召回率不变。
仅报告召回率时四个模型中有三个排名相同，而引入干净对照误报率后差距拉开 79 个百分点。
Can LLMs Catch a Rigged Backtest? A Clean-Control Calibration Benchmark
Backtest auditing is a calibration problem: high flaw recall is not useful when the model falsely flags matched clean strategies. We build a 96-item paired benchmark in which every flawed backtest has a clean control that holds strategy, dates, code style, labels, and reporting scaffold fixed while changing one methodology detail. A deterministic scorer separates flaw recall, clean-control false positives, evidence localization, and fix relevance. Over 1440 cached audits from four text endpoints, the primary DeepSeek auditor reaches 100.0\% closed and clean-aware code recall, but open prompts over-flag 93.8\% of clean code controls, and clean-aware all-three specificity is 87.5\% even where recall saturates. A clean-aware warning drops DeepSeek code false positives from 20.8\% (95\% CI 11.7--34.3) to 0.0\% (0.0--7.4) at unchanged recall, while the budget anchor still flags 38/48 clean controls under the same prompt. Reporting recall alone would rank three of these four models identically; reporting the clean-control rate separates them by 79 points.
本条情报归属 AITOP「论文」核心栏目，重点聚焦中国 AI 产业与国产大模型生态。在跨区域产业观察坐标中，该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」「中美大模型技术代差与商业化对照」研究主线，反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括：DeepSeek、LLM、回测审计、量化金融，读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明：本文原始信息发布自一手权威信源「arXiv: DeepSeek」（Tier-1 一手发布），具备高确定性参考价值。
