Johns Hopkins 论文：训练数据中单次出现的事实越多，LLM 科学发现率越高
Johns Hopkins 把幻觉理论往前推了一步：训练里只见过一次的事实越多，模型越能发现新东西。想做科学模型的可以看看数据构造建议。
Johns Hopkins 的新论文发现，LLM 的科学发现能力与训练数据中'单次出现事实'（monofacts）的占比正相关。作者在 gpt-oss-20b 和 gemma-3-12b-it 上用材料和 DNA 数据集微调，将 monofact 占比从 0% 提到 100%，发现模型对训练中未见过的材料或 DNA 序列的精确复现率随占比稳定上升，且在 2 个模型家族上均成立。同时训练 3 或 5 个 epoch 会把单次出现变成重复，使这一趋势消失。论文将早前把 monofact 与幻觉绑定的理论扩展为：monofact 同时决定模型能发现多少新东西。
Strange findings here.
New Johns Hopkins paper finds that LLMs make more scientific discoveries when more of their training facts appear only once.
discovery rates rose steadily as the share of once-seen facts in training went from 0% to 100%, for 2 model families.
Discovery and hallucination both mean producing something the model never saw in training. Earlier theory tied hallucination to "monofacts", facts seen exactly once, and treated that unseen space as mostly false.
The authors extend that theory to show monofacts also set how much a model can discover. They fine-tuned gpt-oss-20b and gemma-3-12b-it on materials and DNA datasets, varying the monofact share from 0% to 100%.
Discovery, scored as exactly recovering materials or DNA sequences held out from training, rose steadily with the monofact share in every setting. Training for 3 or 5 epochs, which turns single sightings into repeats, made the trend disappear.
When building science models, keep rare one-off examples and avoid repeating data, and expect more hallucinations to come with more discoveries.
