arXiv 论文：表格机器学习的数据扩展规律研究，规模达千万级训练样本
On Data Scaling in Tabular ML: More Data, Better Generalization?
arXiv 上这篇表格 ML 论文拿 8 个数据集、最多约 1000 万样本做数据扩展实验，比较 GBDT、MLP 和表格基础模型谁吃数据，做表格建模的可以看看结论。
这篇 arXiv 论文组装了 8 个真实世界的表格数据集，训练样本最多约 1000 万，用来研究训练数据规模扩展的规律。实验发现 GBDT 和基于 MLP 的模型呈现相似的扩展趋势，更多数据和算法改进都能带来收益；表格基础模型的扩展行为更分化，新版本在更大上下文下受益更稳定。论文指出在分布偏移下，分布内和分布外性能在小规模时同步提升，但在大规模时可能分道扬镳。用与评估条件一致的验证集做模型选择时，MLP 在数据增多后一般能保持或提升性能，而基础模型把样本放进上下文虽能改善表现，却无法阻止随训练数据增多出现的性能退化。
On Data Scaling in Tabular ML: More Data, Better Generalization?
Recent work in tabular machine learning has emphasized the importance of diverse and realistic evaluation settings. The niche of large-scale (1M+) data, however, remains under-explored. To study training-data scaling, we assemble eight real-world tabular datasets with up to around ten million training examples, retaining the natural distribution shifts of the tasks and using random-split evaluation as a control. We find that GBDTs and MLP-based models show broadly similar scaling trends, with gains from both additional training data and algorithmic improvements. Tabular foundation models show more varied scaling behavior, with recent versions benefiting more consistently from larger contexts even under distribution shift. However, scaling training data under distribution shift introduces new challenges: in-distribution and out-of-distribution performance mostly improve together at smaller scales but can diverge at larger scales. We examine this through MLP capacity sweeps and comparisons of the validation distributions used for tuning and early stopping in MLPs and GBDTs. We also compare common practices of using validation data, contrasting its use for model selection in MLP pipelines with using it for in-context fitting in tabular foundation model pipelines. Under stronger shifts, MLPs selected using validation representative of the evaluation conditions generally maintain or improve performance as training data grows. In contrast, including such examples in context improves foundation-model performance, but does not prevent deterioration with increasing training size in our experiments.