研究显示 Gemini 和 DeepSeek 在合金性能预测上超越 SteelBERT 等专用模型
Quantifying out-of-distribution data severity with Wasserstein distances: benchmarking large language models for alloy property prediction
这篇论文拿 Gemini 和 DeepSeek 跟材料学的 SteelBERT 掰手腕，结果通用模型赢了，还只用 5% 高质量数据就能打平全量数据，做材料预测的值得看看方法细节。
arXiv 论文对 Gemini、DeepSeek V3.1 等通用大模型在 Ti、钢材、高熵合金和 Al 性能预测上的表现进行了基准测试，结果显示其准确率超过 SteelBERT、MatSciBERT 和 SciBERT。研究提出混合方法：将 CatBoost 或 SteelBERT 的数值先验作为提示输入 DeepSeek V3.1 或 Gemini-2.5-Flash，让 LLM 充当推理修正器，准确率超过先验模型单独使用的效果。基于 Wasserstein 距离的相似度分析表明数据质量重于数量：只用训练集中 5% 的高相似度参考数据即可达到用 100% 训练集的性能。
Quantifying out-of-distribution data severity with Wasserstein distances: benchmarking large language models for alloy property prediction
As general-purpose Large Language Models (LLMs), such as Gemini and DeepSeek, exhibit increasingly advanced reasoning capabilities, a critical question arises: how do their property prediction abilities compare to those of domain-specific pretrained materials models? In this study, we evaluate predictions for Ti, steels, high-entropy alloy (HEA) and Al, demonstrating that general-purpose LLMs surprisingly outperform SteelBERT, MatSciBERT, and SciBERT. To further enhance performance, we introduce a hybrid approach that utilizes numerical priors from a prior model, such as CatBoost or SteelBERT, to guide the general LLMs (Deepseek V3.1, Gemini-2.5-Flash). By inputting these initial estimates from a prior model as prompts, the LLM acts as a reasoning corrector, achieving accuracy superior to the prior model and "unguided mode". Furthermore, we investigate the impact of data strategy using a similarity-based metric, revealing that data quality dominates quantity; using merely 5% of high-similarity reference data of training set as input yields performance similar to the results of using 100% of training set. Additionally, the LLMs demonstrate excellent uncertainty calibration, where self-reported "high" confidence strongly correlates with minimal deviation from ground truth. Our findings suggest that synergizing general-purpose LLMs with prior model numerical baselines and high-quality data curation creates a cost-effective and accurate paradigm for materials discovery, reducing the need for expensive domain-specific pretraining.
