Scaling Law 的理论基础：一份三步推导的技术笔记
Foundations of the Scaling Law
想搞懂 Scaling Law 为什么长那样的人看这篇：它把损失下降拆成几何、谱、优化三步推导，emergence 和 grokking 也讲清楚了。
arXiv 论文 2610.27789 从三个角度解释经验 Scaling Law：几何近似把空间分辨率、内在维度和平滑度联系到函数误差；谱分析用特征可见性、协方差特征值和目标对齐描述数据增加如何削减未解释的预测质量；约束优化给出参数与数据的配比及训练-推理权衡。文中区分了评分阈值与学习动态来解释 emergence 和 grokking，也把测试时扩展拆成候选覆盖与选中答案准确率两部分。SE(3) 案例量化了商空间近似增益的维度依赖，并比较了视觉与分子预测中不同坐标下的算力分配。
Foundations of the Scaling Law
Empirical scaling laws relate validation loss to model size, training exposure and compute, providing a quantitative basis for allocating resources. This technical note explains these relationships through three linked analyses. Geometric approximation connects spatial resolution, intrinsic dimension and smoothness to function error; the loss function determines how that error transfers to task risk. Spectral analysis combines feature visibility, covariance eigenvalues and target alignment to describe how additional data reduce unresolved predictive mass. Constrained optimization then gives the parameter-data balance and the training-inference trade-off. We also examine emergence and grokking, separating scoring thresholds from learning dynamics, and test-time scaling, separating candidate coverage from selected-answer accuracy. Applications to vision and molecular prediction show how domain-specific coordinates change the resource comparison. An $\mathrm{SE}(3)$ case study quantifies the dimension dependence of quotient-space approximation gains and compares optimized compute frontiers, alongside separate analyses of conformational coverage and correlated-sample precision.
