A Majorized-Generalized Alternating Direction Method of Multipliers for Convex Composite Programming

The linearly constrained convex composite programming problems whose objective function contains two blocks with each block being the form of nonsmooth+smooth arises frequently in multiple fields of applications. If both of the smooth terms are quadratic, this problem can be solved efficiently by using the symmetric Gaussian-Seidel (sGS) technique based proximal alternating direction method of multipliers (ADMM). However, in the non-quadratic case, the sGS technique can not be used any more, which leads to the separable structure of nonsmooth+smooth had to be ignored. In this paper, we present a generalized ADMM and particularly use a majorization technique to make the corresponding subproblems more amenable to efficient computations. Under some appropriate conditions, we prove its global convergence for the relaxation factor in (0, 2). We apply the algorithm to solve a kind of simulated convex composite optimization problems and a type of sparse inverse covariance matrix estimation problems which illustrates that the effectiveness of the algorithm are obvious.

[1]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[2]  R. Glowinski,et al.  Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de Dirichlet non linéaires , 1975 .

[3]  Shiqian Ma,et al.  An Extragradient-Based Alternating Direction Method for Convex Minimization , 2017, Found. Comput. Math..

[4]  John Wright,et al.  Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization , 2009, NIPS.

[5]  D. Gabay.  Applications of the method of multipliers to variational inequalities , 1983 .

[6]  Kim-Chuan Toh,et al.  A block symmetric Gauss–Seidel decomposition theorem for convex composite quadratic programming and its applications , 2017, Mathematical Programming.

[7]  Kim-Chuan Toh,et al.  A Schur complement based semi-proximal ADMM for convex quadratic conic programming and extensions , 2014, Mathematical Programming.

[8]  Paul Tseng,et al.  Hankel Matrix Rank Minimization with Applications to System Identification and Realization , 2013, SIAM J. Matrix Anal. Appl..

[9]  Kim-Chuan Toh,et al.  A Majorized ADMM with Indefinite Proximal Terms for Linearly Constrained Convex Composite Optimization , 2014, SIAM J. Optim..

[10]  Andy M. Yip,et al.  Total Variation Image Restoration: Overview and Recent Developments , 2006, Handbook of Mathematical Models in Computer Vision.

[11]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[12]  Shiqian Ma,et al.  A Block Successive Upper-Bound Minimization Method of Multipliers for Linearly Constrained Convex Optimization , 2014, Math. Oper. Res..

[13]  Yunhai Xiao,et al.  An efficient algorithm for sparse inverse covariance matrix estimation based on dual formulation , 2018, Comput. Stat. Data Anal..

[14]  Junfeng Yang,et al.  A Proximal Point Algorithm for Log-Determinant Optimization with Group Lasso Regularization , 2013, SIAM J. Optim..

[15]  Kim-Chuan Toh,et al.  An efficient inexact symmetric Gauss–Seidel based majorized ADMM for high-dimensional convex composite conic programming , 2015, Mathematical Programming.

[16]  R. Rockafellar.  Monotone Operators and the Proximal Point Algorithm , 1976 .

[17]  Liang Chen,et al.  A generalized alternating direction method of multipliers with semi-proximal terms for convex composite conic programming , 2015, Mathematical Programming Computation.

[18]  H. H. Rachford,et al.  On the numerical solution of heat conduction problems in two and three space variables , 1956 .

[19]  Ying Cui,et al.  On the Convergence Properties of a Majorized Alternating Direction Method of Multipliers for Linearly Constrained Convex Optimization Problems with Coupled Objective Functions , 2016, J. Optim. Theory Appl..