ETD：从 Helmholtz 自由能视角检测大模型预训练数据
Detecting Pretraining Data in Large Language Models from a Free-Energy Perspective
想知道一段文字有没有被模型偷学过？这篇论文的 ETD 用熵校正加自由能检测预训练数据，AUROC 提升最多 3.5%。
一篇 arXiv 论文提出 ETD（Energy Transfer Detection）方法，用于判断某段文本是否出现在大语言模型的预训练数据中。论文指出仅依赖似然的检测器在预测损失与预测熵的联合空间中使用水平边界，会把可预测的非成员误判为成员。ETD 改用倾斜边界，用预测熵校正预测损失，该校正后的分数可解释为 Helmholtz 自由能。实验显示 ETD 将平均 AUROC 提升最多 3.5%，TPR@5%FPR 提升最多 5.1%，并在多种设置下保持稳健。
Detecting Pretraining Data in Large Language Models from a Free-Energy Perspective
Detecting pretraining data in large language models is challenging because high likelihood can reflect either training exposure or strong generalization. In the joint space of prediction loss and predictive entropy, a likelihood-only detector uses a horizontal boundary and can mistake predictable non-members for members. Motivated by this, we introduce an inclined boundary that evaluates prediction loss relative to predictive entropy. Our analysis shows that entropy correction can preserve the expected membership signal while reducing its variance, thereby improving standardized member--non-member separation. We further extend the mean--variance analysis to the more general setting with a nonzero mean entropy gap. Interestingly, this entropy-adjusted score admits a Helmholtz free-energy interpretation, leading to Energy Transfer Detection (ETD), which views pretraining data detection from a macroscopic residual free-energy transfer perspective. Extensive experiments show that ETD achieves the best average detection performance, improving average AUROC by up to 3.5\% and TPR@5\%FPR by up to 5.1\%, while remaining robust across diverse settings.