A Dynamical Approach to an Inertial Forward-Backward Algorithm for Convex Minimization

We introduce a new class of forward-backward algorithms for structured convex minimization problems in Hilbert spaces. Our approach relies on the time discretization of a second-order differential system with two potentials and Hessian-driven damping, recently introduced in [H. Attouch, P.-E. Mainge, and P. Redont, Differ. Equ. Appl., 4 (2012), pp. 27--65]. This system can be equivalently written as a first-order system in time and space, each of the two constitutive equations involving one (and only one) of the two potentials. Its time dicretization naturally leads to the introduction of forward-backward splitting algorithms with inertial features. Using a Liapunov analysis, we show the convergence of the algorithm under conditions enlarging the classical step size limitation. Then, we specialize our results to gradient-projection algorithms and give some illustrations of sparse signal recovery and feasibility problems.

[1]  Marc Teboulle,et al.  Gradient-based algorithms with applications to signal-recovery problems , 2010, Convex Optimization in Signal Processing and Communications.

[2]  P. L. Combettes,et al.  Hilbertian convex feasibility problem: Convergence of projection methods , 1997 .

[3]  Heinz H. Bauschke,et al.  On Projection Algorithms for Solving Convex Feasibility Problems , 1996, SIAM Rev..

[4]  R. Rockafellar.  Monotone Operators and the Proximal Point Algorithm , 1976 .

[5]  M. Solodov,et al.  A Hybrid Approximate Extragradient – Proximal Point Algorithm Using the Enlargement of a Maximal Monotone Operator , 1999 .

[6]  Gregory B. Passty.  Ergodic convergence to a zero of the sum of monotone operators in Hilbert space , 1979 .

[7]  Patrick L. Combettes,et al.  Hard-constrained inconsistent signal feasibility problems , 1999, IEEE Trans. Signal Process..

[8]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .

[9]  H. Attouch,et al.  An Inertial Proximal Method for Maximal Monotone Operators via Discretization of a Nonlinear Oscillator with Damping , 2001 .

[10]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .

[11]  Paul Tseng,et al.  A Modified Forward-backward Splitting Method for Maximal Monotone Mappings 1 , 1998 .

[12]  C. Byrne,et al.  A unified treatment of some iterative algorithms in signal processing and image reconstruction , 2003 .

[13]  S. Sorin,et al.  Evolution equations for maximal monotone operators: asymptotic analysis in continuous and discrete time , 2009, 0905.1270.

[14]  C. Byrne,et al.  Iterative oblique projection onto convex sets and the split feasibility problem , 2002 .

[15]  P. Tseng.  Applications of splitting algorithm to decomposition in convex programming and variational inequalities , 1991 .

[16]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[17]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[18]  H. Attouch,et al.  A second-order differential system with hessian-driven damping; application to non-elastic shock laws , 2012 .

[19]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[20]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[21]  Ronald E. Bruck.  Asymptotic convergence of nonlinear contraction semigroups in Hilbert space , 1975 .

[22]  Paul Tseng,et al.  Further applications of a splitting algorithm to decomposition in variational inequalities and convex programming , 1990, Math. Program..

[23]  Patrick L. Combettes,et al.  A Monotone+Skew Splitting Model for Composite Monotone Inclusions in Duality , 2010, SIAM J. Optim..

[24]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[25]  H. Brezis.  Opérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert , 1973 .

[26]  Y. Nesterov.  A method for solving the convex programming problem with convergence rate O(1/k^2) , 1983 .

[27]  Patrick L. Combettes,et al.  A Parallel Splitting Method for Coupled Monotone Inclusions , 2009, SIAM J. Control. Optim..

[28]  O. Nelles,et al.  An Introduction to Optimization , 1996, IEEE Antennas and Propagation Magazine.

[29]  Marc Teboulle,et al.  A proximal-based decomposition method for convex minimization problems , 1994, Math. Program..

[30]  A. M. Geoffrion.  Proper efficiency and the theory of vector maximization , 1968 .

[31]  Felipe Alvarez,et al.  Weak Convergence of a Relaxed and Inertial Hybrid Projection-Proximal Point Algorithm for Maximal Monotone Operators in Hilbert Space , 2003, SIAM J. Optim..

[32]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[33]  Juan Peypouquet,et al.  Coupling Forward-Backward with Penalty Schemes and Parallel Splitting for Constrained Variational Inequalities , 2011, SIAM J. Optim..

[34]  A. Antipin,et al.  MINIMIZATION OF CONVEX FUNCTIONS ON CONVEX SETS BY MEANS OF DIFFERENTIAL EQUATIONS , 2003 .

[35]  Benar Fux Svaiter,et al.  Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods , 2013, Math. Program..

[36]  Yair Censor,et al.  A multiprojection algorithm using Bregman projections in a product space , 1994, Numerical Algorithms.

[37]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[38]  J. Bolte,et al.  A second-order gradient-like dissipative dynamical system with Hessian-driven damping.: Application to optimization and mechanics , 2002 .

[39]  P. L. Combettes,et al.  Solving monotone inclusions via compositions of nonexpansive averaged operators , 2004 .