论文多源确认

DeepSeek V4.1 Flash-1017 独立安全评估:34 项基准对照 GPT-6 等闭源模型

Evaluating DeepSeek V4.1 Flash-1017 for loss of control and harmful manipulation risks

精选理由

有人在 DeepSeek V4.1 Flash-1017 发布后独立测了它的失控和操纵风险,还和 GPT-6、Gemini 3.8 摆在一起对比,结论有具体建议,做安全评估的可以看看。

一项第三方研究参照欧盟通用 AI 行为准则,对 DeepSeek V4.1 Flash-1017 进行了独立安全评估,覆盖评估完整性、失控风险和有害操纵三大类共 34 项基准。该模型相比 V4 一代在安全性上有明显改进,在部分维度接近 GPT-6 Astra、GPT-5.6 Sol 和 Gemini 3.8 Flash 等闭源系统。评估发现轻微版本更新可能造成安全性回退,建议开发者将安全评估纳入增量发布流程。报告同时指出仅靠最终输出和结束状态检查无法可靠检测失控行为,需要在推理轨迹和动作日志层面做过程级可观测。

原文 · arXiv: DeepSeek

Evaluating DeepSeek V4.1 Flash-1017 for loss of control and harmful manipulation risks

As frontier language models grow increasingly capable and autonomous, independent third-party evaluations are critical for characterizing their systemic safety risks. While closed-source releases are routinely accompanied by comprehensive system cards, open-weight models receive less scrutiny regarding frontier risks. To address this gap, we present an independent safety evaluation of DeepSeek V4.1 Flash-1017, the latest version of V4.1 Flash that departs from V4 with a new architecture, scaled-up training, and strong agentic capabilities. Drawing on the EU General-Purpose AI Code of Practice, we evaluate the model across 34 benchmarks, including 5 in-house suites, spanning evaluation integrity, loss of control, and harmful manipulation. Conforming to the AEF-1 standard, we compare V4.1 Flash-1017 against V4.1 Flash, earlier DeepSeek V4 models, and leading closed-source systems (GPT-6 Astra, GPT-5.6 Sol, and Gemini 3.8 Flash). Overall, V4.1 Flash-1017 shows substantial safety improvements over the V4 generation and approaches or matches leading closed-source baselines on several dimensions, while still exhibiting notable behaviors that warrant continued attention. Building on our findings, we highlight three key observations and recommendations. First, because safety can shift and even regress across minor updates, developers should integrate safety evaluation into incremental releases. Second, as final-turn outputs and end-state checks alone cannot reliably detect loss-of-control behaviors, deployments require process-level observability across both reasoning traces and action logs. Third, because public benchmarks elicit strong evaluation awareness and face data contamination risks, developing private benchmarks is essential for trustworthy frontier evaluation.