论文

pdLIP:用学习到的预条件子加速原始-对偶内点法

Learned Preconditioning for a Primal-Dual Interior-Point Method

精选理由

优化圈的朋友看看这篇:把预条件子交给神经网络学,内点法解非线性规划能省掉 Hessian 和牛顿求解,warm start 直接砍掉六成多迭代。

arXiv 论文提出 pdLIP,一个结合学习预条件与 pdProj 投影搜索的原始-对偶内点法(IPM),用于求解光滑非线性规划。共享的逐坐标循环网络预测正对角预条件子,省去 Hessian 计算和牛顿方程组求解,只保留可 GPU 并行的一阶运算。训练采用自监督方式,损失基于惩罚障碍 merit 函数和扰动最优性条件残差,无需目标方向或预计算解。在四类 200 维凸与非凸约束问题上,warm start 比 cold start 减少 63-67% 的 pdProj 精化迭代,且在 1000 变量的箱约束 QP 上依然有效,应用涵盖投资组合优化、支持向量机和非线性控制。

原文 · arXiv cs.LG

Learned Preconditioning for a Primal-Dual Interior-Point Method

Interior-point methods (IPMs) are among the most widely used algorithms for constrained optimization, yet their Newton-based search directions require costly second-order information and large linear-system solves. Learning to optimize offers cheaper updates learned from data, but the singular behavior of logarithmic barriers near constraint boundaries makes IPMs highly sensitive to perturbations, complicating both warm starting and learning reliable updates. We introduce pdLIP, an IPM for smooth nonlinear programs that integrates learned preconditioning with pdProj, an all-shifted primal-dual projected-search IPM. A shared coordinate-wise recurrent network predicts a positive diagonal preconditioner that scales the right-hand side of the reduced Newton system for the primal step, and the remaining slack and multiplier directions are recovered analytically. The learned iterations avoid Hessian evaluations and Newton-system solves, using only first-order and coordinate-wise operations amenable to GPU parallelization. Training is self-supervised, with a loss based on a penalty-barrier merit function and the residual of perturbed optimality conditions, requiring neither target directions nor precomputed solutions. Primal and dual shifts mitigate the barrier's sensitivity to perturbations near constraint boundaries, enabling effective warm starting. Across four classes of 200-dimensional convex and nonconvex constrained problems, pdLIP warm starts reduce pdProj refinement iterations by 63-67% compared with cold starts at the same KKT residual tolerance of $10^{-8}$, with negligible warm-start generation cost relative to the subsequent pdProj solve. Improvements persist on box-constrained QPs with 1000 variables and extend to applications including portfolio optimization, support vector machines, and a nonlinear control example.