Penalty schemes with inertial effects for monotone inclusion problems

We introduce a penalty term-based splitting algorithm with inertial effects designed for solving monotone inclusion problems involving the sum of maximally monotone operators and the convex normal cone to the (nonempty) set of zeros of a monotone and Lipschitz continuous operator. We show weak ergodic convergence of the generated sequence of iterates to a solution of the monotone inclusion problem, provided a condition expressed via the Fitzpatrick function of the operator describing the underlying set of the normal cone is verified. Under strong monotonicity assumptions we can even show strong nonergodic convergence of the iterates. This approach constitutes the starting point for investigating from a similar perspective monotone inclusion problems involving linear compositions of parallel-sum operators and, further, for the minimization of a complexly structured convex objective function subject to the set of minima of another convex and differentiable function.

[1]  R. Rockafellar,et al.  On the maximal monotonicity of subdifferential mappings. , 1970 .

[2]  R. Rockafellar.  On the maximality of sums of nonlinear monotone operators , 1970 .

[3]  R. Rockafellar.  Monotone Operators and the Proximal Point Algorithm , 1976 .

[4]  I. Ekeland,et al.  Convex analysis and variational problems , 1976 .

[5]  Willi Hock,et al.  Lecture Notes in Economics and Mathematical Systems , 1981 .

[6]  S. Fitzpatrick.  Representing monotone operators by convex functions , 1988 .

[7]  O. Nelles,et al.  An Introduction to Optimization , 1996, IEEE Antennas and Propagation Magazine.

[8]  Felipe Alvarez,et al.  On the Minimizing Property of a Second Order Dissipative System in Hilbert Spaces , 2000, SIAM J. Control. Optim..

[9]  Paul Tseng,et al.  A Modified Forward-backward Splitting Method for Maximal Monotone Mappings 1 , 1998 .

[10]  H. Attouch,et al.  An Inertial Proximal Method for Maximal Monotone Operators via Discretization of a Nonlinear Oscillator with Damping , 2001 .

[11]  C. Zălinescu.  Convex analysis in general vector spaces , 2002 .

[12]  Benar Fux Svaiter,et al.  Maximal Monotone Operators, Convex Functions and a Special Family of Enlargements , 2002 .

[13]  A. Moudafi,et al.  Convergence of a splitting inertial proximal method for monotone operators , 2003 .

[14]  Felipe Alvarez,et al.  Weak Convergence of a Relaxed and Inertial Hybrid Projection-Proximal Point Algorithm for Maximal Monotone Operators in Hilbert Space , 2003, SIAM J. Optim..

[15]  M. Nikolova.  An Algorithm for Total Variation Minimization and Applications , 2004 .

[16]  Hristo S. Sendov,et al.  Fitzpatrick functions: inequalities, examples, and remarks on a problem by S. Fitzpatrick , 2005 .

[17]  J. Borwein.  Maximal Monotonicity via Convex Analysis , 2006 .

[18]  S. Simons.  From Hahn-Banach to monotonicity , 2008 .

[19]  R. Boţ,et al.  An Application of the Bivariate Inf-Convolution Formula to Enlargements of Monotone Operators , 2008 .

[20]  Paul-Emile Maingé,et al.  Convergence of New Inertial Proximal Methods for DC Programming , 2008, SIAM J. Optim..

[21]  P. Maingé.  Convergence theorems for inertial KM-type algorithms , 2008 .

[22]  H. Attouch,et al.  Asymptotic behavior of coupled dynamical systems with multiscale aspects , 2009, 0904.0397.

[23]  R. Boţ,et al.  Conjugate Duality in Convex Optimization , 2010 .

[24]  J. Borwein,et al.  Convex Functions: Constructions, Characterizations and Counterexamples , 2010 .

[25]  Patrick L. Combettes,et al.  A Monotone+Skew Splitting Model for Composite Monotone Inclusions in Duality , 2010, SIAM J. Optim..

[26]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[27]  A. Cabot,et al.  Asymptotics for Some Proximal-like Method Involving Inertia and Memory Aspects , 2011 .

[28]  P. L. Combettes,et al.  Primal-Dual Splitting Algorithm for Solving Inclusions with Mixtures of Composite, Lipschitzian, and Parallel-Sum Type Monotone Operators , 2011, Set-Valued and Variational Analysis.

[29]  Juan Peypouquet,et al.  Prox-Penalization and Splitting Methods for Constrained Variational Problems , 2011, SIAM J. Optim..

[30]  Juan Peypouquet,et al.  Coupling Forward-Backward with Penalty Schemes and Parallel Splitting for Constrained Variational Inequalities , 2011, SIAM J. Optim..

[31]  Juan Peypouquet,et al.  Coupling the Gradient Method with a General Exterior Penalization Scheme for Convex Minimization , 2012, J. Optim. Theory Appl..

[32]  R. Boţ,et al.  Forward-Backward and Tseng’s Type Penalty Schemes for Monotone Inclusion Problems , 2013, 1306.0352.

[33]  Radu Ioan Bot,et al.  Convergence Analysis for a Primal-Dual Monotone + Skew Splitting Algorithm with Applications to Total Variation Minimization , 2012, Journal of Mathematical Imaging and Vision.

[34]  Juan Peypouquet,et al.  Forward–Backward Penalty Scheme for Constrained Convex Minimization Without Inf-Compactness , 2013, J. Optim. Theory Appl..

[35]  R. Boţ,et al.  A Hybrid Proximal-Extragradient Algorithm with Inertial Effects , 2014, 1407.0214.

[36]  Juan Peypouquet,et al.  A Dynamical Approach to an Inertial Forward-Backward Algorithm for Convex Minimization , 2014, SIAM J. Optim..

[37]  R. Boţ,et al.  An inertial alternating direction method of multipliers , 2014, 1404.4582.

[38]  R. Boţ,et al.  A Tseng’s Type Penalty Scheme for Solving Inclusion Problems Involving Linearly Composed and Parallel-Sum Type Monotone Operators , 2014 .

[39]  Thomas Brox,et al.  iPiano: Inertial Proximal Algorithm for Nonconvex Optimization , 2014, SIAM J. Imaging Sci..

[40]  Radu Ioan Bot,et al.  Backward Penalty Schemes for Monotone Inclusion Problems , 2014, J. Optim. Theory Appl..

[41]  R. Boţ,et al.  Approaching the solving of constrained variational inequalities via penalty term-based dynamical systems , 2015, 1503.01871.

[42]  Caihua Chen,et al.  A General Inertial Proximal Point Algorithm for Mixed Variational Inequality Problem , 2015, SIAM J. Optim..

[43]  Radu Ioan Bot,et al.  Inertial Douglas-Rachford splitting for monotone inclusion problems , 2014, Appl. Math. Comput..

[44]  Shiqian Ma,et al.  Inertial Proximal ADMM for Linearly Constrained Separable Convex Optimization , 2015, SIAM J. Imaging Sci..

[45]  Radu Ioan Bot,et al.  An inertial forward-backward-forward primal-dual splitting algorithm for solving monotone inclusion problems , 2014, Numerical Algorithms.

[46]  David Stutz.  IPIANO : INERTIAL PROXIMAL ALGORITHM FOR NON-CONVEX OPTIMIZATION , 2016 .

[47]  Radu Ioan Bot,et al.  An inertial forward–backward algorithm for the minimization of the sum of two nonconvex functions , 2014, EURO J. Comput. Optim..

[48]  Radu Ioan Bot,et al.  An Inertial Tseng’s Type Proximal Algorithm for Nonsmooth and Nonconvex Optimization Problems , 2014, J. Optim. Theory Appl..