On the Moreau-Yosida Regularization of the Vector k-Norm Related Functions

Matrix optimization problems (MOPs) involving the Ky Fan $k$-norm arise frequently from many applications. In order to design algorithms to solve large scale MOPs involving the Ky Fan $k$-norm, we need to understand the first and second order properties of the Moreau--Yosida regularization of the Ky Fan $k$-norm function and the indicator function of the Ky Fan $k$-norm ball. According to the general theory on spectral functions, in this paper we shall conduct a thorough study on the Moreau--Yosida regularization of the vector $k$-norm function and the indicator function of the vector $k$-norm ball. In particular, we show that the proximal mappings associated with these two vector $k$-norm related functions both admit fast and analytically computable solutions. Moreover, we propose algorithms of low computational cost to compute the directional derivatives of these two proximal mappings and then completely characterize their Frechet differentiability. The work here thus builds the fundamental tools needed...

[1]  Defeng Sun,et al.  A Quadratically Convergent Newton Method for Computing the Nearest Correlation Matrix , 2006, SIAM J. Matrix Anal. Appl..

[2]  Nathan Srebro,et al.  Sparse Prediction with the $k$-Support Norm , 2012, NIPS.

[3]  Farid Alizadeh,et al.  Interior Point Methods in Semidefinite Programming with Applications to Combinatorial Optimization , 1995, SIAM J. Optim..

[4]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[5]  Xinyuan Zhao,et al.  A SEMISMOOTH NEWTON-CG AUGMENTED LAGRANGIAN METHOD FOR LARGE SCALE LINEAR AND CONVEX QUADRATIC SDPS , 2009 .

[6]  Arkadi Nemirovski,et al.  Lectures on modern convex optimization - analysis, algorithms, and engineering applications , 2001, MPS-SIAM series on optimization.

[7]  Kim-Chuan Toh,et al.  An introduction to a class of matrix cone programming , 2012, Mathematical Programming.

[8]  Stephen P. Boyd,et al.  Fastest Mixing Markov Chain on a Graph , 2004, SIAM Rev..

[9]  Yan Gao,et al.  Calibrating Least Squares Semidefinite Programming with Equality and Inequality Constraints , 2009, SIAM J. Matrix Anal. Appl..

[10]  G. Alistair Watson.  Linear best approximation using a class of polyhedral norms , 2005, Numerical Algorithms.

[11]  J. J. Moré,et al.  Quasi-Newton updates with bounds , 1987 .

[12]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[13]  A. S. Lewis,et al.  Derivatives of Spectral Functions , 1996, Math. Oper. Res..

[14]  R. Plemmons,et al.  Structured low rank approximation , 2003 .

[15]  Jeffery L. Kennington,et al.  A polynomially bounded algorithm for a singly constrained quadratic program , 1980, Math. Program..

[16]  Gábor Pataki,et al.  On the Rank of Extreme Matrices in Semidefinite Programs and the Multiplicity of Optimal Eigenvalues , 1998, Math. Oper. Res..

[17]  Jong-Shi Pang,et al.  Newton's Method for B-Differentiable Equations , 1990, Math. Oper. Res..

[18]  Franz Rendl,et al.  Regularization Methods for Semidefinite Programming , 2009, SIAM J. Optim..

[19]  Melvyn Sim,et al.  Robust linear optimization under general norms , 2004, Oper. Res. Lett..

[20]  R. Jackson.  Inequalities , 2007, Algebra for Parents.

[21]  Philip Wolfe,et al.  Validation of subgradient optimization , 1974, Math. Program..

[22]  A. Haraux.  How to differentiate the projection on a convex set in Hilbert space. Some applications to variational inequalities , 1977 .

[23]  P. Wolfe,et al.  The minimization of certain nondifferentiable sums of eigenvalues of symmetric matrices , 1975 .

[24]  Defeng Sun.  Calibrating Least Squares Covariance Matrix Problems with Equality and Inequality Constraints , 2008 .

[25]  Defeng Sun,et al.  Semismooth Matrix-Valued Functions , 2002, Math. Oper. Res..

[26]  R. Tyrrell Rockafellar,et al.  Augmented Lagrangians and Applications of the Proximal Point Algorithm in Convex Programming , 1976, Math. Oper. Res..

[27]  Stephen M. Robinson,et al.  Strongly Regular Generalized Equations , 1980, Math. Oper. Res..

[28]  丸山 徹.  Convex Analysisの二,三の進展について , 1977 .

[29]  Defeng Sun,et al.  A Majorized Penalty Approach for Calibrating Rank Constrained Correlation Matrix Problems , 2010 .

[30]  Franz Rendl,et al.  A Boundary Point Method to Solve Semidefinite Programs , 2006, Computing.

[31]  Michael L. Overton,et al.  Optimality conditions and duality theory for minimizing sums of the largest eigenvalues of symmetric matrices , 2015, Math. Program..

[32]  A. Lewis.  The Convex Analysis of Unitarily Invariant Matrix Functions , 1995 .

[33]  R. Rockafellar.  Monotone Operators and the Proximal Point Algorithm , 1976 .

[34]  Xiaoqi Yang,et al.  Semismoothness of Spectral Functions , 2003, SIAM J. Matrix Anal. Appl..

[35]  Bastian Goldlücke,et al.  Variational Analysis , 2014, Computer Vision, A Reference Guide.

[36]  Stephen P. Boyd,et al.  Fastest Mixing Markov Chain on Graphs with Symmetries , 2007, SIAM J. Optim..

[37]  Adrian S. Lewis,et al.  Twice Differentiable Spectral Functions , 2001, SIAM J. Matrix Anal. Appl..

[38]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[39]  P. Brucker.  Review of recent development: An O( n) algorithm for quadratic knapsack problems , 1984 .

[40]  N. Maculan,et al.  An O(n) Algorithm for Projecting a Vector on the Intersection of a Hyperplane and a Box in Rn , 2003 .

[41]  G. Alistair Watson,et al.  On matrix approximation problems with Ky Fank norms , 1993, Numerical Algorithms.

[42]  Krzysztof C. Kiwiel,et al.  Breakpoint searching algorithms for the continuous quadratic knapsack problem , 2007, Math. Program..

[43]  Panos M. Pardalos,et al.  An algorithm for a singly constrained class of quadratic programs subject to upper and lower bounds , 1990, Math. Program..

[44]  Wotao Yin,et al.  Alternating direction augmented Lagrangian methods for semidefinite programming , 2010, Math. Program. Comput..

[45]  Ding Chao,et al.  AN INTRODUCTION TO A CLASS OF MATRIX OPTIMIZATION PROBLEMS , 2012 .

[46]  Kim-Chuan Toh,et al.  A Newton-CG Augmented Lagrangian Method for Semidefinite Programming , 2010, SIAM J. Optim..

[47]  J. Moreau.  Proximité et dualité dans un espace hilbertien , 1965 .

[48]  Kim-Chuan Toh,et al.  Solving semidefinite-quadratic-linear programs using SDPT3 , 2003, Math. Program..