Solving Nuclear Norm Regularized and Semidefinite Matrix Least Squares Problems with Linear Equality Constraints

We introduce a partial proximal point algorithm for solving nuclear norm regularized and semidefinite matrix least squares problems with linear equality constraints. For the inner subproblems, we show that the positive definiteness of the generalized Hessian of the objective function for the inner subproblems is equivalent to the constraint nondegeneracy of the corresponding primal problem, which is a key property for applying a semismooth Newton-CG method to solve the inner subproblems efficiently. Numerical experiments on large scale matrix least squares problems arising from low rank matrix approximation, as well as regularized kernel estimation and Euclidean distance matrix completion problems in molecular conformation, show that our algorithm is efficient and robust.

[1]  R. Tyrrell Rockafellar,et al.  Convex Analysis , 1970, Princeton Landmarks in Mathematics and Physics.

[2]  Kim-Chuan Toh,et al.  A Newton-CG Augmented Lagrangian Method for Semidefinite Programming , 2010, SIAM J. Optim..

[3]  R. Rockafellar.  Monotone Operators and the Proximal Point Algorithm , 1976 .

[4]  P. L. Combettes,et al.  Primal-Dual Splitting Algorithm for Solving Inclusions with Mixtures of Composite, Lipschitzian, and Parallel-Sum Type Monotone Operators , 2011, Set-Valued and Variational Analysis.

[5]  Kim-Chuan Toh,et al.  A partial proximal point algorithm for nuclear norm regularized matrix least squares problems , 2014, Math. Program. Comput..

[6]  Gene H. Golub,et al.  Matrix computations , 1983 .

[7]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[8]  Liqun Qi,et al.  A nonsmooth version of Newton's method , 1993, Math. Program..

[9]  Yong-Jin Liu,et al.  An implementable proximal point algorithmic framework for nuclear norm minimization , 2012, Math. Program..

[10]  Stephen J. Wright,et al.  Framework for kernel regularization with application to protein clustering. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[11]  C. Ha.  A generalization of the proximal point algorithm , 1987, 26th IEEE Conference on Decision and Control.

[12]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[13]  S. Yun,et al.  An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems , 2009 .

[14]  Karl Löwner.  Über monotone Matrixfunktionen , 1934 .

[15]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[16]  Yang Zhe,et al.  A STUDY ON NONSYMMETRIC MATRIX-VALUED FUNCTIONS , 2009 .

[17]  Shiqian Ma,et al.  Fixed point and Bregman iterative methods for matrix rank minimization , 2009, Math. Program..

[18]  Kim-Chuan Toh,et al.  A Distributed SDP Approach for Large-Scale Noisy Anchor-Free Graph Realization with Applications to Molecular Conformation , 2008, SIAM J. Sci. Comput..

[19]  S. Yun,et al.  An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems , 2009 .

[20]  Stephen P. Boyd,et al.  A rank minimization heuristic with application to minimum order system approximation , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[21]  Yan Gao,et al.  Calibrating Least Squares Semidefinite Programming with Equality and Inequality Constraints , 2009, SIAM J. Matrix Anal. Appl..

[22]  R. Glowinski,et al.  Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de Dirichlet non linéaires , 1975 .

[23]  R. Tyrrell Rockafellar,et al.  Augmented Lagrangians and Applications of the Proximal Point Algorithm in Convex Programming , 1976, Math. Oper. Res..

[24]  Henry Wolkowicz,et al.  Solving Euclidean Distance Matrix Completion Problems Via Semidefinite Programming , 1999, Comput. Optim. Appl..

[25]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[26]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[27]  Yifan Hu,et al.  A maxent-stress model for graph layout , 2012, PacificVis.

[28]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[29]  G. Stewart,et al.  A generalization of the Eckart-Young-Mirsky matrix approximation theorem , 1987 .

[30]  B. Schölkopf,et al.  Graph Laplacian Regularization for Large-Scale Semidefinite Programming , 2007 .

[31]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[32]  Laurent Condat,et al.  A Primal–Dual Splitting Method for Convex Optimization Involving Lipschitzian, Proximable and Linear Composite Terms , 2012, Journal of Optimization Theory and Applications.

[33]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[34]  Sung-Hou Kim,et al.  Global mapping of the protein structure space and application in structure-based inference of protein function. , 2005, Proceedings of the National Academy of Sciences of the United States of America.