AdamX：一种结合余弦相似度的优化器
AdamX: Cosine similarity meets gradient descent
朋友A推荐朋友B试试AdamX这个新优化器，说它结合了余弦相似度，能控制更新幅度，在训练中表现不错。
我们介绍了一种名为AdamX的新优化器，它将余弦相似度作为自适应机制来控制更新幅度。该方法在多个基准数据集和架构上实现了有竞争力的收敛率。作者提供了代码和实验。
AdamX: Cosine similarity meets gradient descent
We introduce AdamX, a first-order optimizer that incorporates cosine similarity as an adaptive mechanism for controlling update magnitudes. The proposed method is scalable, model-agnostic, and straightforward to integrate into existing training pipelines. We further introduce a variance rectification scheme that promotes smoother optimization during the early stages of training. Overall, we provide empirical evidence that AdamX achieves competitive convergence rates across a range of benchmark datasets and architectures. Performance is evaluated in terms of the number of epochs required to reach predefined performance thresholds under a fixed hyperparameter budget. Code and Experiments available at: https://github.com/FranciscoCaldas/adamX.
