HotPOT：用异构物理专家算子 Transformer 做大规模 PDE 预训练
HotPOT: Operator Transformer with Heterogeneous Physics Experts for Large-Scale PDE Pre-training
一篇把物理方程直接写进专家结构的 PDE 论文，17M 小模型反而打赢更大的 MoE-POT，做科学计算的可以看看代码。
HotPOT 是一个面向大规模 PDE 预训练的算子 Transformer，采用共享编码器，每个 PDE 家族配备一个神经-数值专家，将可学习预测器与无可训练参数的固定数值更新配对。在 5 个 PDE 家族、6 种设置的实验中，HotPOT 在 24 组同规模对比中全部优于 MoE-POT，滚动误差平均降低 50.7%。仅 17M 参数的模型也能超过大得多的 MoE-POT 模型，且在未见物理条件下保持零样本预测能力。代码已在 GitHub 开源。
HotPOT: Operator Transformer with Heterogeneous Physics Experts for Large-Scale PDE Pre-training
Large-scale PDE pre-training trains a single model on the data from multiple partial differential equations (PDEs). However, adding heterogeneous PDEs with different governing equations, physical variables, and variable interactions does not always improve predictive performance across tasks. Existing PDE mixture-of-experts models mitigate cross-task interference through learned routing, but typically use homogeneous expert architectures, making it difficult to directly reflect differences in state evolution across equations. We introduce HotPOT, an operator transformer with heterogeneous physics experts for large-scale PDE pre-training. HotPOT uses a shared encoder to capture information common across PDEs, while each PDE family uses a neural-numerical expert that pairs a learnable predictor with a fixed numerical update containing no trainable parameters. Each predictor outputs the quantities required by its paired numerical update, which then computes the next state. This design allows different PDEs to share a common encoder while retaining their own learnable predictors and numerical updates. Experiments across five PDE families and six settings show that HotPOT outperforms MoE-POT in all 24 matched-scale comparisons before and after fine-tuning, reducing rollout error by 50.7% on average. Notably, the 17M model also surpasses substantially larger MoE-POT models, and the performance gains extend to zero-shot prediction under unseen physical conditions. Code is available at https://github.com/cruiseresearchgroup/HotPOT-code.