eKSVD：将核奇异值分解扩展到多数据源的联合非线性特征学习
Kernel Singular Value Decomposition with Extension to Multiple Data Sources
一篇把 KSVD 推广到多数据源的理论论文，还用神经网络做显式特征映射，做核方法或特征学习方向的可以看看推导。
arXiv 论文 2610.03216 提出扩展 KSVD，得到 eKSVD，可在多个数据源之间进行联合非线性特征学习。该方法在原始问题中对各数据源关联的投影联合求解，捕捉最大信息并纳入成对耦合。通过对偶优化及 KKT 条件，eKSVD 将 KSVD 中 Lanczos 分解定理的移位特征值问题推广到多源情形。论文还给出基于协方差的框架，用神经网络作为显式特征映射，数值实验显示在处理多数据源时优于基于 Mercer 核的方法。
Kernel Singular Value Decomposition with Extension to Multiple Data Sources
Kernel Singular Value Decomposition (KSVD) learns a pair of singular vectors w.r.t. an asymmetric kernel matrix, which can be induced by two data sources, e.g., the queries and keys in self-attention or the rows and columns of a given matrix. In this work, we extend KSVD to multiple data sources, namely eKSVD, which conducts joint nonlinear feature learning upon asymmetric kernels. In the primal formulation, the projections associated with each data source are jointly learned to capture maximal information, while incorporating pair-wise couplings. With the Lagrangian and its Karush-Kuhn-Tucker (KKT) conditions, the optimization in the dual leads to a generalization of the shifted eigenvalue problem in Lanczos decomposition theorem of KSVD. Further, a covariance-based framework is derived together with using neural networks (NNs) for explicit feature mappings, complementary to the kernel-based interpretation and optimization. Numerical experiments verify the effectiveness of our eKSVD compared to methods based on Mercer kernels for tackling multiple data sources, and our innovation of deploying NNs demonstrates great flexibility for kernel methods.
