论文

AlignedHPO:梯度对齐引导的可微损失超参数优化框架

AlignedHPO: Efficient Surrogate-Guided Loss Hyperparameter Optimization

精选理由

训练里调 Focal Loss、ArcFace margin 这类损失参数太费时间的话,这篇给了个基于梯度对齐的自动方案,比 implicit differentiation 快近 12 倍,长尾分类和人脸识别都有实验结果,做训练优化的工程团队可以直接看。

学术论文 AlignedHPO 提出一种针对可微损失超参数的自动调优框架,面向 Focal Loss 权重、ArcFace margin 这类通常靠人工或黑盒搜索调整的参数。方法上用 Jacobian-vector products 降低超梯度计算的时间与内存开销,并以可微指标代理(surrogate)作为验证目标,引导优化对齐到具体评估标准,同时通过归一化对齐和梯度平滑处理梯度量级歧义与随机不稳定。

正文

实验覆盖长尾分类和开集人脸识别两个任务,在 Generalized Focal Loss 与 ArcFace 上验证了框架的适配能力。结果显示其性能与 implicit differentiation 相当或更优,超梯度计算速度提升近 12 倍。对需要在大规模数据上反复调损失函数超参数的工程团队来说,12 倍的速度差异意味着原本按天计的调参流程可以压缩到小时级。

原文 · arXiv cs.AI

AlignedHPO: Efficient Surrogate-Guided Loss Hyperparameter Optimization

Modern deep learning losses involve multiple hyperparameters, such as weights, scales, and margins, typically tuned manually or through expensive black-box optimization despite their differentiability. We propose AlignedHPO, an efficient framework for tuning differentiable loss hyperparameters using gradient alignment and surrogate validation objectives. Our method employs Jacobian-vector products to reduce hypergradient computation time and memory, while differentiable metric surrogates guide optimization toward task-specific evaluation criteria. We address gradient magnitude ambiguity and stochastic instability through normalized alignment and gradient smoothing. Experiments with generalized Focal Loss and ArcFace demonstrate effective adaptation across long-tailed classification and open-set face recognition. AlignedHPO achieves competitive or improved performance compared to implicit differentiation, with nearly 12x faster hypergradient computation. These results highlight the potential of efficient surrogate-guided loss hyperparameter optimization.

产业坐标与信源核验

本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:超参数优化、ArcFace、Focal Loss、损失函数,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「arXiv cs.AI」(@Ivan Karpukhin, Andrey Savchenko),经 AITOP 多维评分管线结构化提炼。