Inertial proximal best approximation primal-dual algorithm

We propose a new modified primal–dual proximal best approximation method for solving convex not necessarily differentiable optimization problems. The novelty of the method relies on introducing memory by taking into account iterates computed in previous steps in the formulas defining current iterate. To this end we consider projections onto intersections of halfspaces generated on the basis of the current as well as the previous iterates. To calculate these projections we are using recently obtained closed-form expressions for projectors onto polyhedral sets. The resulting algorithm with memory inherits strong convergence properties of the original best approximation proximal primal–dual algorithm. Additionally, we compare our algorithm with the original (non-inertial) one with the help of the so called attraction property defined below. Extensive numerical experimental results on image reconstruction problems illustrate the advantages of including memory into the original algorithm.

[1]  Quoc Tran-Dinh,et al.  A new splitting method for solving composite monotone inclusions involving parallel-sum operators , 2015, 1505.07946.

[2]  Patrick L. Combettes,et al.  Best Approximation from the Kuhn-Tucker Set of Composite Monotone Inclusions , 2014, 1401.8005.

[3]  A. Moudafi,et al.  AN APPROXIMATE INERTIAL PROXIMAL METHOD USING THE ENLARGEMENT OF A MAXIMAL MONOTONE OPERATOR , 2003 .

[4]  J. Pesquet,et al.  A Parallel Inertial Proximal Optimization Method , 2012 .

[5]  Radu Ioan Bot,et al.  An inertial forward-backward-forward primal-dual splitting algorithm for solving monotone inclusion problems , 2014, Numerical Algorithms.

[6]  H. Attouch,et al.  An Inertial Proximal Method for Maximal Monotone Operators via Discretization of a Nonlinear Oscillator with Damping , 2001 .

[7]  Patrick L. Combettes,et al.  A Parallel Splitting Method for Coupled Monotone Inclusions , 2009, SIAM J. Control. Optim..

[8]  Thomas Brox,et al.  iPiasco: Inertial Proximal Algorithm for Strongly Convex Optimization , 2015, Journal of Mathematical Imaging and Vision.

[9]  Benar Fux Svaiter,et al.  General Projective Splitting Methods for Sums of Maximal Monotone Operators , 2009, SIAM J. Control. Optim..

[10]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[11]  Patrick L. Combettes,et al.  Asynchronous block-iterative primal-dual decomposition methods for monotone inclusions , 2015, Mathematical Programming.

[12]  A. Moudafi A hybrid inertial projection-proximal method for variational inequalities. , 2004 .

[13]  Caihua Chen,et al.  A General Inertial Proximal Point Algorithm for Mixed Variational Inequality Problem , 2015, SIAM J. Optim..

[14]  R. Boţ,et al.  An inertial alternating direction method of multipliers , 2014, 1404.4582.

[15]  L. Rosasco,et al.  A stochastic inertial forward–backward splitting algorithm for multivariate monotone inclusions , 2015, 1507.00848.

[16]  Radu Ioan Bot,et al.  A Douglas-Rachford Type Primal-Dual Method for Solving Inclusions with Mixtures of Composite and Parallel-Sum Type Monotone Operators , 2012, SIAM J. Optim..

[17]  Nikos Komodakis,et al.  Playing with Duality: An overview of recent primal?dual approaches for solving large-scale optimization problems , 2014, IEEE Signal Process. Mag..

[18]  Shiqian Ma,et al.  Inertial Proximal ADMM for Linearly Constrained Separable Convex Optimization , 2015, SIAM J. Imaging Sci..

[19]  Heinz H. Bauschke A Note on the Paper by Eckstein and Svaiter on "General Projective Splitting Methods for Sums of Maximal Monotone Operators" , 2009, SIAM J. Control. Optim..

[20]  R. Rockafellar Convex Analysis: (pms-28) , 1970 .

[21]  Patrick L. Combettes,et al.  Strong Convergence of Block-Iterative Outer Approximation Methods for Convex Optimization , 2000, SIAM J. Control. Optim..

[22]  Radu Ioan Bot,et al.  A double smoothing technique for solving unconstrained nondifferentiable convex optimization problems , 2012, Computational Optimization and Applications.

[23]  Zhao Yang Dong,et al.  A fast dual proximal-gradient method for separable convex optimization with linear coupled constraints , 2016, Comput. Optim. Appl..

[24]  Thomas Brox,et al.  iPiano: Inertial Proximal Algorithm for Nonconvex Optimization , 2014, SIAM J. Imaging Sci..

[25]  Dirk A. Lorenz,et al.  An Inertial Forward-Backward Algorithm for Monotone Inclusions , 2014, Journal of Mathematical Imaging and Vision.

[26]  Paul-Emile Maingé,et al.  NUMERICAL APPROACH TO A STATIONARY SOLUTION OF A SECOND ORDER DISSIPATIVE DYNAMICAL SYSTEM , 2002 .

[27]  Patrick L. Combettes,et al.  Fejér Monotonicity in Convex Optimization , 2009, Encyclopedia of Optimization.

[28]  Guo-ji Tang,et al.  Strong convergence of a splitting projection method for the sum of maximal monotone operators , 2014, Optim. Lett..

[29]  Benar Fux Svaiter,et al.  Forcing strong convergence of proximal point iterations in a Hilbert space , 2000, Math. Program..

[30]  P. Maingé Inertial Iterative Process for Fixed Points of Certain Quasi-nonexpansive Mappings , 2007 .

[31]  Krzysztof E. Rutkowski Closed-Form Expressions for Projectors onto Polyhedral Sets in Hilbert Spaces , 2017, SIAM J. Optim..

[32]  P. L. Combettes,et al.  Primal-Dual Splitting Algorithm for Solving Inclusions with Mixtures of Composite, Lipschitzian, and Parallel-Sum Type Monotone Operators , 2011, Set-Valued and Variational Analysis.

[33]  Christopher Hendrich Proximal Splitting Methods in Nonsmooth Convex Optimization , 2014 .

[34]  A. Moudafi,et al.  Convergence of a splitting inertial proximal method for monotone operators , 2003 .

[35]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[36]  Ming Yan,et al.  ARock: an Algorithmic Framework for Asynchronous Parallel Coordinate Updates , 2015, SIAM J. Sci. Comput..

[37]  Pham Ky Anh,et al.  Modified hybrid projection methods for finding common solutions to variational inequality problems , 2017, Comput. Optim. Appl..

[38]  Niao He,et al.  Mirror Prox algorithm for multi-term composite minimization and semi-separable problems , 2013, Computational Optimization and Applications.

[39]  Nan-Jing Huang,et al.  Strong convergence of a splitting proximal projection method for the sum of two maximal monotone operators , 2012, Oper. Res. Lett..

[40]  Patrick L. Combettes,et al.  Solving Composite Monotone Inclusions in Reflexive Banach Spaces by Constructing Best Bregman Approximations from Their Kuhn-Tucker Set , 2015, 1505.00362.

[41]  Radu Ioan Bot,et al.  On the convergence rate improvement of a primal-dual splitting algorithm for solving monotone inclusion problems , 2013, Mathematical Programming.

[42]  Émilie Chouzenoux,et al.  A block coordinate variable metric forward–backward algorithm , 2016, Journal of Global Optimization.

[43]  Felipe Alvarez,et al.  Weak Convergence of a Relaxed and Inertial Hybrid Projection-Proximal Point Algorithm for Maximal Monotone Operators in Hilbert Space , 2003, SIAM J. Optim..

[44]  Radu Ioan Bot,et al.  Inertial Douglas-Rachford splitting for monotone inclusion problems , 2014, Appl. Math. Comput..

[45]  Naihua Xiu,et al.  Modified Extragradient Method for Variational Inequalities and Verification of Solution Existence , 2003 .

[46]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[47]  Benar Fux Svaiter,et al.  A family of projective splitting methods for the sum of two maximal monotone operators , 2007, Math. Program..

[48]  P. Maingé Regularized and Inertial algorithms for common fixed points of nonlinear operators , 2008 .

[49]  Jonathan Eckstein,et al.  A Simplified Form of Block-Iterative Operator Splitting and an Asynchronous Algorithm Resembling the Multi-Block Alternating Direction Method of Multipliers , 2017, J. Optim. Theory Appl..

[50]  Patrick L. Combettes,et al.  Solving Coupled Composite Monotone Inclusions by Successive Fejér Approximations of their Kuhn-Tucker Set , 2013, SIAM J. Optim..

[51]  Antonin Chambolle,et al.  On the ergodic convergence rates of a first-order primal–dual algorithm , 2016, Math. Program..

[52]  P. Maingé Convergence theorems for inertial KM-type algorithms , 2008 .

[53]  Pierre Moulin,et al.  Local and global convergence of a general inertial proximal splitting scheme for minimizing composite functions , 2016, Comput. Optim. Appl..

[54]  Teemu Pennanen,et al.  Dualization of Generalized Equations of Maximal Monotone Type , 1999, SIAM J. Optim..