The space decomposition theory for a class of eigenvalue optimizations

In this paper we study optimization problems involving eigenvalues of symmetric matrices. One of the difficulties with numerical analysis of such problems is that the eigenvalues, considered as functions of a symmetric matrix, are not differentiable at those points where they coalesce. Here we apply the $\mathcal{U}$-Lagrangian theory to a class of D.C. functions (the difference of two convex functions): the arbitrary eigenvalue function λi, with affine matrix-valued mappings, where λi is a D.C. function. We give the first-and second-order derivatives of ${\mathcal{U}}$-Lagrangian in the space of decision variables Rm when transversality condition holds. Moreover, an algorithm framework with quadratic convergence is presented. Finally, we present an application: low rank matrix optimization; meanwhile, list its $\mathcal{VU}$ decomposition results.

[1]  Pierre Apkarian,et al.  A Spectral Quadratic-SDP Method with Applications to Fixed-Order H2 and H∞ Synthesis , 2004, Eur. J. Control.

[2]  Alexander Shapiro,et al.  On Eigenvalue Optimization , 1995, SIAM J. Optim..

[3]  Joaquim Júdice,et al.  Efficient DC programming approaches for the asymmetric eigenvalue complementarity problem , 2013, Optim. Methods Softw..

[4]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[5]  Robert Mifflin,et al.  On VU-theory for Functions with Primal-Dual Gradient Structure , 2000, SIAM J. Optim..

[6]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[7]  J. Hiriart-Urruty,et al.  Sensitivity analysis of all eigenvalues of a symmetric matrix , 1995 .

[8]  Michael L. Overton,et al.  Large-Scale Optimization of Eigenvalues , 1990, SIAM J. Optim..

[9]  Ilya V. Kolmanovsky,et al.  Predictive energy management of a power-split hybrid electric vehicle , 2009, 2009 American Control Conference.

[10]  François Oustry,et al.  A second-order bundle method to minimize the maximum eigenvalue function , 2000, Math. Program..

[11]  M. Overton,et al.  The reduced density matrix method for electronic structure calculations and the role of three-index representability conditions. , 2004, The Journal of chemical physics.

[12]  Michael L. Overton,et al.  Optimality conditions and duality theory for minimizing sums of the largest eigenvalues of symmetric matrices , 2015, Math. Program..

[13]  R. Rockafellar.  Convex Analysis: (pms-28) , 1970 .

[14]  Le Thi Hoai An,et al.  The DC (Difference of Convex Functions) Programming and DCA Revisited with DC Models of Real World Nonconvex Optimization Problems , 2005, Ann. Oper. Res..

[15]  Pierre Apkarian,et al.  Partially Augmented Lagrangian Method for Matrix Inequality Constraints , 2004, SIAM J. Optim..

[16]  Michael L. Overton,et al.  Second Derivatives for Optimizing Eigenvalues of Symmetric Matrices , 1995, SIAM J. Matrix Anal. Appl..

[17]  Stephen Becker,et al.  Quantum state tomography via compressed sensing. , 2009, Physical review letters.

[18]  Alexander Shapiro,et al.  First and second order analysis of nonlinear semidefinite programs , 1997, Math. Program..

[19]  R. Weismantel,et al.  A Semidefinite Programming Approach to the Quadratic Knapsack Problem , 2000, J. Comb. Optim..

[21]  Steven J. Cox,et al.  Extremal eigenvalue problems for two-phase conductors , 1996 .

[22]  Robert Mifflin,et al.  A -algorithm for convex minimization , 2005, Math. Program..

[23]  Le Thi Hoai An,et al.  A DC Programming Approach for Sparse Eigenvalue Problem , 2010, ICML.

[24]  Le Thi Hoai An,et al.  A D.C. Optimization Algorithm for Solving the Trust-Region Subproblem , 1998, SIAM J. Optim..

[25]  David Gross,et al.  Recovering Low-Rank Matrices From Few Coefficients in Any Basis , 2009, IEEE Transactions on Information Theory.

[26]  Yu. S. Ledyaev,et al.  Nonsmooth analysis and control theory , 1998 .

[27]  Yuan Lu,et al.  A superlinear space decomposition algorithm for constrained nonsmooth convex program , 2010, J. Comput. Appl. Math..

[28]  J. Frédéric Bonnans,et al.  Perturbation Analysis of Optimization Problems , 2000, Springer Series in Operations Research.

[29]  Stephen P. Boyd,et al.  Semidefinite Programming , 1996, SIAM Rev..

[30]  C. Lemaréchal,et al.  THE U -LAGRANGIAN OF A CONVEX FUNCTION , 1996 .

[31]  Pierre Apkarian,et al.  Spectral bundle methods for non-convex maximum eigenvalue functions: second-order methods , 2005, Math. Program..

[32]  M. Fazel,et al.  Reweighted nuclear norm minimization with application to system identification , 2010, Proceedings of the 2010 American Control Conference.

[33]  Benjamin Recht,et al.  A Simpler Approach to Matrix Completion , 2009, J. Mach. Learn. Res..

[34]  Emmanuel J. Candès,et al.  The Power of Convex Relaxation: Near-Optimal Matrix Completion , 2009, IEEE Transactions on Information Theory.

[35]  Robert Mifflin,et al.  𝒱𝒰-smoothness and proximal point results for some nonconvex functions , 2004, Optim. Methods Softw..

[36]  N. Kikuchi,et al.  Solutions to shape and topology eigenvalue optimization problems using a homogenization method , 1992 .

[37]  François Oustry,et al.  The U-Lagrangian of the Maximum Eigenvalue Function , 1999, SIAM J. Optim..