Convergence of a Hybrid Projection-Proximal Point Algorithm Coupled with Approximation Methods in Convex Optimization

In order to minimize a closed convex function that is approximated by a sequence of better behaved functions, we investigate the global convergence of a general hybrid iterative algorithm, which consists of an inexact relaxed proximal point step followed by a suitable orthogonal projection onto a hyperplane. The latter permits to consider a fixed relative error criterion for the proximal step. We provide various sets of conditions ensuring the global convergence of this algorithm. The analysis is valid for nonsmooth data in infinite-dimensional Hilbert spaces. Some examples are presented, focusing on penalty/barrier methods in convex programming. We also show that some results can be adapted to the zero-finding problem for a maximal monotone operator.

[1]  B. Lemaire.  About the Convergence of the Proximal Method , 1992 .

[2]  M. Solodov,et al.  A UNIFIED FRAMEWORK FOR SOME INEXACT PROXIMAL POINT ALGORITHMS , 2001 .

[3]  Roberto Cominetti,et al.  Primal and dual convergence of a proximal point exponential penalty method for linear programming , 2002, Math. Program..

[4]  Clóvis C. Gonzaga,et al.  Path-Following Methods for Linear Programming , 1992, SIAM Rev..

[5]  Hédy Attouch,et al.  Viscosity Solutions of Minimization Problems , 1996, SIAM J. Optim..

[6]  Michael C. Ferris,et al.  Operator-Splitting Methods for Monotone Affine Variational Inequalities, with a Parallel Application to Optimal Control , 1998, INFORMS J. Comput..

[7]  N. V. Tretyakov,et al.  Modified Lagrangians and Monotone Maps in Optimization , 1996 .

[8]  A. Auslender.  Numerical methods for nondifferentiable convex optimization , 1987 .

[9]  Fabiana Zama,et al.  Iterative methods for ill-posed problems and semiconvergent sequences , 2006 .

[10]  I. Konnov.  Combined Relaxation Methods for Variational Inequalities , 2000 .

[11]  N. Megiddo.  Pathways to the optimal set in linear programming , 1989 .

[12]  B. Martinet.  Brève communication. Régularisation d'inéquations variationnelles par approximations successives , 1970 .

[13]  Roberto Cominetti,et al.  Asymptotic analysis of the exponential penalty trajectory in linear programming , 1994, Math. Program..

[14]  P. Alart,et al.  Penalization in non-classical convex programming via variational convergence , 1991, Math. Program..

[15]  Gian-Carlo Rota.  Opérateurs maximaux monotones: H. Brézis, North-Holland, 1983, 183 pp. , 1985 .

[16]  F. Alvarez.  Absolute minimizer in convex programming by exponential penalty. , 2000 .

[17]  E. G. Gol'shtein,et al.  Modified Lagrangians in Convex Programming and their Generalizations , 1979 .

[18]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[19]  Jean Charles Gilbert,et al.  Numerical Optimization: Theoretical and Practical Aspects , 2003 .

[20]  E. G. Golʹshteĭn,et al.  Modified Lagrangians and monotone maps in optimization , 1996 .

[21]  R. Cominetti.  Coupling the Proximal Point Algorithm with Approximation Methods , 1997 .

[22]  Z. Opial.  Weak convergence of the sequence of successive approximations for nonexpansive mappings , 1967 .

[23]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.

[24]  R. Cominetti.  Nonlinear Averages and Convergence of Penalty Trajectories in Convex Programming , 1999 .

[25]  H. Brezis.  Opérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert , 1973 .

[26]  Khalil Mouallif,et al.  Une méthode de pénalisation exponentielle associée à une régularisation proximale , 1987 .

[27]  Variational metric and exponential penalization , 1990 .

[28]  Osman Güer.  On the convergence of the proximal point algorithm for convex minimization , 1991 .

[29]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[30]  T. Champion.  Tubularity and Asymptotic Convergence of Penalty Trajectories in Convex Programming , 2002, SIAM J. Optim..

[31]  Claude Lemaréchal,et al.  Convergence of some algorithms for convex minimization , 1993, Math. Program..

[32]  M. Solodov,et al.  A hybrid projection-proximal point algorithm. , 1998 .

[33]  A. Auslender,et al.  Penalty-proximal methods in convex programming , 1987 .

[34]  G. Sonnevend.  An "analytical centre" for polyhedrons and new classes of global algorithms for linear (smooth, convex) programming , 1986 .

[35]  Roberto Cominetti,et al.  Coupling General Penalty Schemes for Convex Programming with the Steepest Descent and the Proximal Point Algorithm , 2002, SIAM J. Optim..

[36]  Alexander Kaplan,et al.  Proximal Methods in View of Interior-Point Strategies , 1995, Universität Trier, Mathematik/Informatik, Forschungsbericht.

[37]  Felipe Alvarez,et al.  Weak Convergence of a Relaxed and Inertial Hybrid Projection-Proximal Point Algorithm for Maximal Monotone Operators in Hilbert Space , 2003, SIAM J. Optim..

[38]  Roberto Cominetti,et al.  Asymptotic Analysis for Penalty and Barrier Methods in Convex and Linear Programming , 1997, Math. Oper. Res..

[39]  Alfredo N. Iusem,et al.  On the need for hybrid steps in hybrid proximal point methods , 2001, Oper. Res. Lett..

[40]  H. Attouch,et al.  L p approximation of varitional problems in L 1 and L , 1999 .

[41]  R. Tyrrell Rockafellar,et al.  Augmented Lagrangians and Applications of the Proximal Point Algorithm in Convex Programming , 1976, Math. Oper. Res..

[42]  Michael A. Saunders,et al.  A Practical Interior-Point Method for Convex Programming , 1995, SIAM J. Optim..

[43]  M. J. D. Powell,et al.  Nonlinear Programming—Sequential Unconstrained Minimization Techniques , 1969 .

[44]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[45]  D. Bertsekas.  Approximation procedures based on the method of multipliers , 1977 .

[46]  Alexandre Cabot,et al.  Proximal Point Algorithm Controlled by a Slowly Vanishing Term: Applications to Hierarchical Minimization , 2005, SIAM J. Optim..

[47]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[48]  Renato D. C. Monteiro,et al.  On the Existence and Convergence of the Central Path for Convex Programming and Some Duality Results , 1998, Comput. Optim. Appl..

[49]  R. Rockafellar.  Monotone Operators and the Proximal Point Algorithm , 1976 .

[50]  H. Attouch,et al.  A Dynamical Approach to Convex Minimization Coupling Approximation with the Steepest Descent Method , 1996 .

[51]  Anders Forsgren,et al.  Interior Methods for Nonlinear Optimization , 2002, SIAM Rev..

[52]  B. Lemaire,et al.  Convergence of diagonally stationary sequences in convex optimization , 1994 .

[53]  H. Attouch.  Variational convergence for functions and operators , 1984 .

[54]  J. Frédéric Bonnans,et al.  Numerical Optimization: Theoretical and Practical Aspects (Universitext) , 2006 .

[55]  M. Solodov,et al.  A UNIFIED FRAMEWORK FOR SOME INEXACT PROXIMAL POINT ALGORITHMS , 2001 .

[56]  B. Lemaire.  Coupling optimization methods and variational convergence , 1988 .