An alternate minimization method beyond positive definite proximal regularization: convergence and complexity

In this paper, an alternate minimization method beyond positive definite proximal regularization is introduced for solving linearly constrained separable convex optimization problems. The proposed method can be interpreted as the predictioncorrection method from the perspective of variational inequalities. The convergence of the proposed method is established without strong convexity. Finally, the iteration complexity of the proposed method is also derived in the ergodic sense.

[1]  Liya Liu,et al.  Strong convergence of an extragradient-like algorithm involving pseudo-monotone mappings , 2019, Numerical Algorithms.

[2]  Bingsheng He,et al.  Convergence Study on the Symmetric Version of ADMM with Larger Step Sizes , 2016, SIAM J. Imaging Sci..

[3]  Xiangfeng Wang,et al.  Asynchronous Distributed ADMM for Large-Scale Optimization—Part II: Linear Convergence Analysis and Numerical Performance , 2015, IEEE Transactions on Signal Processing.

[4]  Bingsheng He,et al.  A Strictly Contractive Peaceman-Rachford Splitting Method for Convex Programming , 2014, SIAM J. Optim..

[5]  Zhixun Su,et al.  Linearized alternating direction method with parallel splitting and adaptive penalty for separable convex programs in machine learning , 2013, Machine Learning.

[6]  Xiaoming Yuan,et al.  An ADM-based splitting method for separable convex programming , 2013, Comput. Optim. Appl..

[7]  Junfeng Yang,et al.  Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization , 2012, Math. Comput..

[8]  Bingsheng He,et al.  On the O(1/n) Convergence Rate of the Douglas-Rachford Alternating Direction Method , 2012, SIAM J. Numer. Anal..

[9]  Xiaoming Yuan,et al.  A Note on the Alternating Direction Method of Multipliers , 2012, J. Optim. Theory Appl..

[10]  Raymond H. Chan,et al.  Alternating Direction Method for Image Inpainting in Wavelet Domains , 2011, SIAM J. Imaging Sci..

[11]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[12]  Stanley Osher,et al.  A Unified Primal-Dual Algorithm Framework Based on Bregman Iteration , 2010, J. Sci. Comput..

[13]  Michael K. Ng,et al.  Solving Constrained Total-variation Image Restoration and Reconstruction Problems via Alternating Direction Methods , 2010, SIAM J. Sci. Comput..

[14]  Junfeng Yang,et al.  A Fast Alternating Direction Method for TVL1-L2 Signal Reconstruction From Partial Fourier Data , 2010, IEEE Journal of Selected Topics in Signal Processing.

[15]  X. P. Ding,et al.  Auxiliary Principle and Algorithm for Mixed Equilibrium Problems and Bilevel Mixed Equilibrium Problems in Banach Spaces , 2010 .

[16]  Junfeng Yang,et al.  A New Alternating Minimization Algorithm for Total Variation Image Reconstruction , 2008, SIAM J. Imaging Sci..

[17]  Bingsheng He,et al.  A new inexact alternating directions method for monotone variational inequalities , 2002, Math. Program..

[18]  Patrice Marcotte,et al.  Co-Coercivity and Its Role in the Convergence of Iterative Schemes for Solving Variational Inequalities , 1996, SIAM J. Optim..

[19]  R. Tyrrell Rockafellar,et al.  Augmented Lagrangians and Applications of the Proximal Point Algorithm in Convex Programming , 1976, Math. Oper. Res..

[20]  M. Hestenes.  Multiplier and gradient methods , 1969 .

[21]  Luyao Zhang,et al.  A modified inertial projection and contraction algorithms for quasi-variational inequalities , 2019, Applied Set-Valued Analysis and Optimization.

[22]  Xiaoming Yuan,et al.  Positive-indefinite Proximal Augmented Lagrangian Method and Its Application to Full Jacobian Splitting for Multi-block Separable Convex Minimization Problems , 2022 .

[23]  Xiaoming Yuan,et al.  Linearized Alternating Direction Method of Multipliers via Positive-Indefinite Proximal Regularization for Convex Programming , 2016 .

[24]  P. Tseng,et al.  A decomposition property for a class of square matrices , 1991 .

[25]  M. Fortin,et al.  Augmented Lagrangian methods : applications to the numerical solution of boundary-value problems , 1983 .

[26]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[27]  R. Glowinski,et al.  Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de Dirichlet non linéaires , 1975 .

[28]  M. Powell.  A method for nonlinear constraints in minimization problems , 1969 .