The Proximal Alternating Minimization Algorithm for Two-Block Separable Convex Optimization Problems with Linear Constraints

The Alternating Minimization Algorithm has been proposed by Paul Tseng to solve convex programming problems with two-block separable linear constraints and objectives, whereby (at least) one of the components of the latter is assumed to be strongly convex. The fact that one of the subproblems to be solved within the iteration process of this method does not usually correspond to the calculation of a proximal operator through a closed formula affects the implementability of the algorithm. In this paper, we allow in each block of the objective a further smooth convex function and propose a proximal version of the algorithm, which is achieved by equipping the algorithm with proximal terms induced by variable metrics. For suitable choices of the latter, the solving of the two subproblems in the iterative scheme can be reduced to the computation of proximal operators. We investigate the convergence of the proposed algorithm in a real Hilbert space setting and illustrate its numerical performances on two applications in image processing and machine learning.

[1]  P. L. Combettes,et al.  Variable metric quasi-Fejér monotonicity , 2012, 1206.5705.

[2]  Radu Ioan Bot,et al.  Convergence Analysis for a Primal-Dual Monotone + Skew Splitting Algorithm with Applications to Total Variation Minimization , 2012, Journal of Mathematical Imaging and Vision.

[3]  M. Fortin,et al.  On decomposition - coordination methods using an augmented Lagrangian , 1983 .

[4]  Marc Teboulle,et al.  Rate of Convergence Analysis of Decomposition Methods Based on the Proximal Method of Multipliers for Convex Minimization , 2014, SIAM J. Optim..

[5]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[6]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[7]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[8]  R. Boţ,et al.  Conjugate Duality in Convex Optimization , 2010 .

[9]  Radu Ioan Bot,et al.  Fixing and extending some recent results on the ADMM algorithm , 2016, Numerical Algorithms.

[10]  Christopher Hendrich.  Proximal Splitting Methods in Nonsmooth Convex Optimization , 2014 .

[11]  Tony F. Chan,et al.  A General Framework for a Class of First Order Primal-Dual Algorithms for Convex Optimization in Imaging Science , 2010, SIAM J. Imaging Sci..

[12]  Radu Ioan Bot,et al.  On the convergence rate improvement of a primal-dual splitting algorithm for solving monotone inclusion problems , 2013, Mathematical Programming.

[13]  Heinz H. Bauschke,et al.  Fixed-Point Algorithms for Inverse Problems in Science and Engineering , 2011, Springer Optimization and Its Applications.

[14]  M. Fortin,et al.  Augmented Lagrangian methods : applications to the numerical solution of boundary-value problems , 1983 .

[15]  Kristian Bredies,et al.  Preconditioned Douglas-Rachford Splitting Methods for Convex-concave Saddle-point Problems , 2015, SIAM J. Numer. Anal..

[16]  Kristian Bredies,et al.  A Proximal Point Analysis of the Preconditioned Alternating Direction Method of Multipliers , 2017, Journal of Optimization Theory and Applications.

[17]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[18]  Émilie Chouzenoux,et al.  A block coordinate variable metric forward–backward algorithm , 2016, Journal of Global Optimization.

[19]  Paul Tseng,et al.  Hankel Matrix Rank Minimization with Applications to System Identification and Realization , 2013, SIAM J. Matrix Anal. Appl..

[20]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[21]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[22]  Hedy Attouch,et al.  Augmented Lagrangian and Proximal Alternating Direction Methods of Multipliers in Hilbert spaces . Applications to Games , PDE ’ s and Control , 2019 .

[23]  P. Tseng.  Applications of splitting algorithm to decomposition in convex programming and variational inequalities , 1991 .

[24]  Radu Ioan Bot,et al.  ADMM for monotone operators: convergence analysis and rates , 2017, Advances in Computational Mathematics.