A General Inertial Proximal Point Algorithm for Mixed Variational Inequality Problem

In this paper, we first propose a general inertial proximal point algorithm (PPA) for the mixed variational inequality (VI) problem. Based on our knowledge, without stronger assumptions, a convergence rate result is not known in the literature for inertial type PPAs. Under certain conditions, we are able to establish the global convergence and nonasymptotic $O(1/k)$ convergence rate result (under a certain measure) of the proposed general inertial PPA. We then show that both the linearized augmented Lagrangian method (ALM) and the linearized alternating direction method of multipliers (ADMM) for structured convex optimization are applications of a general PPA, provided that the algorithmic parameters are properly chosen. Consequently, global convergence and convergence rate results of the linearized ALM and ADMM follow directly from results existing in the literature. In particular, by applying the proposed inertial PPA for mixed VI to structured convex optimization, we obtain inertial versions of the lin...

[1]  M. J. D. Powell,et al.  A method for nonlinear constraints in minimization problems , 1969 .

[2]  Radu Ioan Bot,et al.  An Inertial Tseng’s Type Proximal Algorithm for Nonsmooth and Nonconvex Optimization Problems , 2014, J. Optim. Theory Appl..

[3]  Xiaoming Yuan,et al.  A proximal point algorithm revisit on the alternating direction method of multipliers , 2013 .

[4]  Junfeng Yang,et al.  Alternating Direction Algorithms for 1-Problems in Compressive Sensing , 2009, SIAM J. Sci. Comput..

[5]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[6]  Z. Opial.  Weak convergence of the sequence of successive approximations for nonexpansive mappings , 1967 .

[7]  Felipe Alvarez,et al.  Weak Convergence of a Relaxed and Inertial Hybrid Projection-Proximal Point Algorithm for Maximal Monotone Operators in Hilbert Space , 2003, SIAM J. Optim..

[8]  H. H. Rachford,et al.  On the numerical solution of heat conduction problems in two and three space variables , 1956 .

[9]  Paul Tseng,et al.  A Modified Forward-backward Splitting Method for Maximal Monotone Mappings 1 , 1998 .

[10]  M. Hestenes.  Multiplier and gradient methods , 1969 .

[11]  Shiqian Ma,et al.  Fixed point and Bregman iterative methods for matrix rank minimization , 2009, Math. Program..

[12]  Junfeng Yang,et al.  Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization , 2012, Math. Comput..

[13]  R. Tyrrell Rockafellar,et al.  Augmented Lagrangians and Applications of the Proximal Point Algorithm in Convex Programming , 1976, Math. Oper. Res..

[14]  R. Boţ,et al.  An inertial alternating direction method of multipliers , 2014, 1404.4582.

[15]  Wotao Yin,et al.  On the o(1/k) Convergence and Parallelization of the Alternating Direction Method of Multipliers , 2013 .

[16]  Xiaoming Yuan,et al.  A Generalized Proximal Point Algorithm and Its Convergence Rate , 2014, SIAM J. Optim..

[17]  Boris Polyak.  Some methods of speeding up the convergence of iteration methods , 1964 .

[18]  David Stutz.  IPIANO : INERTIAL PROXIMAL ALGORITHM FOR NON-CONVEX OPTIMIZATION , 2016 .

[19]  Felipe Alvarez,et al.  On the Minimizing Property of a Second Order Dissipative System in Hilbert Spaces , 2000, SIAM J. Control. Optim..

[20]  Francesco Zirilli,et al.  Algorithm 617: DAFNE: a differential-equations algorithm for nonlinear equations , 1984, ACM Trans. Math. Softw..

[21]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[22]  Damek Davis,et al.  Convergence Rate Analysis of Several Splitting Schemes , 2014, 1406.4834.

[23]  R. Glowinski,et al.  Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de Dirichlet non linéaires , 1975 .

[24]  R. Rockafellar.  Monotone Operators and the Proximal Point Algorithm , 1976 .

[25]  Marc Teboulle,et al.  Rate of Convergence Analysis of Decomposition Methods Based on the Proximal Method of Multipliers for Convex Minimization , 2014, SIAM J. Optim..

[26]  Thomas Brox,et al.  iPiasco: Inertial Proximal Algorithm for Strongly Convex Optimization , 2015, Journal of Mathematical Imaging and Vision.

[27]  Bingsheng He,et al.  On non-ergodic convergence rate of Douglas–Rachford alternating direction method of multipliers , 2014, Numerische Mathematik.

[28]  Radu Ioan Bot,et al.  Inertial Douglas-Rachford splitting for monotone inclusion problems , 2014, Appl. Math. Comput..

[29]  Bingsheng He,et al.  On the O(1/n) Convergence Rate of the Douglas-Rachford Alternating Direction Method , 2012, SIAM J. Numer. Anal..

[30]  Jonathan Eckstein.  Splitting methods for monotone operators with applications to parallel optimization , 1989 .

[31]  Yi Ma,et al.  Compressive principal component pursuit , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[32]  Y. Nesterov.  A method for solving the convex programming problem with convergence rate O(1/k^2) , 1983 .

[33]  Radu Ioan Bot,et al.  An inertial forward-backward-forward primal-dual splitting algorithm for solving monotone inclusion problems , 2014, Numerical Algorithms.

[34]  Juan Peypouquet,et al.  A Dynamical Approach to an Inertial Forward-Backward Algorithm for Convex Minimization , 2014, SIAM J. Optim..

[35]  A. Moudafi,et al.  Approximate inertial proximal methods using the enlargement of maximal monotone operators , 2003 .

[36]  H. Attouch,et al.  An Inertial Proximal Method for Maximal Monotone Operators via Discretization of a Nonlinear Oscillator with Damping , 2001 .

[37]  A. Antipin,et al.  MINIMIZATION OF CONVEX FUNCTIONS ON CONVEX SETS BY MEANS OF DIFFERENTIAL EQUATIONS , 2003 .

[38]  Osman Güler,et al.  New Proximal Point Algorithms for Convex Minimization , 1992, SIAM J. Optim..

[39]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[40]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[41]  Thomas Brox,et al.  iPiano: Inertial Proximal Algorithm for Nonconvex Optimization , 2014, SIAM J. Imaging Sci..

[42]  Dirk A. Lorenz,et al.  An Inertial Forward-Backward Algorithm for Monotone Inclusions , 2014, Journal of Mathematical Imaging and Vision.

[43]  A. Moudafi,et al.  A proximal method for maximal monotone operators via discretization of a first order dissipative dynamical system , 2007 .

[44]  J. Moreau.  Proximité et dualité dans un espace hilbertien , 1965 .

[45]  Ronald E. Bruck.  Asymptotic convergence of nonlinear contraction semigroups in Hilbert space , 1975 .

[46]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[47]  R. Boţ,et al.  A Hybrid Proximal-Extragradient Algorithm with Inertial Effects , 2014, 1407.0214.

[48]  Paul-Emile Maingé,et al.  A new inertial-type hybrid projection-proximal algorithm for monotone inclusions , 2010, Appl. Math. Comput..

[49]  Paul Tseng,et al.  Hankel Matrix Rank Minimization with Applications to System Identification and Realization , 2013, SIAM J. Matrix Anal. Appl..

[50]  B. Martinet.  Brève communication. Régularisation d'inéquations variationnelles par approximations successives , 1970 .