New error bounds and their applications to convergence analysis of iterative algorithms

Abstract.We present two new error bounds for optimization problems over a convex set whose objective function f is either semianalytic or γ-strictly convex, with γ≥1. We then apply these error bounds to analyze the rate of convergence of a wide class of iterative descent algorithms for the aforementioned optimization problem. Our analysis shows that the function sequence {f(xk)} converges at least at the sublinear rate of k-ε for some positive constant ε, where k is the iteration index. Moreover, the distances from the iterate sequence {xk} to the set of stationary points of the optimization problem converge to zero at least sublinearly.

[1]  Wu Li.  Error Bounds for Piecewise Convex Quadratic Programs and Applications , 1995 .

[2]  Paul Tseng,et al.  Error Bound and Convergence Analysis of Matrix Splitting Algorithms for the Affine Variational Inequality Problem , 1992, SIAM J. Optim..

[3]  P. Tseng,et al.  On the linear convergence of descent methods for convex essentially smooth minimization , 1992 .

[4]  J. Warga.  Minimizing Certain Convex Functions , 1963 .

[5]  D. Bertsekas,et al.  Combined Primal–Dual and Penalty Methods for Convex Programming , 1976 .

[6]  D. Bertsekas.  On the Goldstein-Levitin-Polyak gradient projection method , 1974, CDC 1974.

[7]  Alfredo N. Iusem.  On Dual Convergence and the Rate of Primal Convergence of Bregman's Convex Programming Method , 1991, SIAM J. Optim..

[8]  S. Łojasiewicz.  Sur le problème de la division , 1959 .

[9]  Jong-Shi Pang,et al.  Error bounds in mathematical programming , 1997, Math. Program..

[10]  F. Luque.  Asymptotic convergence analysis of the proximal point algorithm , 1984 .

[11]  Z.-Q. Luo,et al.  Error bounds and convergence analysis of feasible descent methods: a general approach , 1993, Ann. Oper. Res..

[12]  Osman Güer.  On the convergence of the proximal point algorithm for convex minimization , 1991 .

[13]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[14]  J. Dunn.  Global and Asymptotic Convergence Rate Estimates for a Class of Projected Gradient Processes , 1981 .

[15]  D. Bertsekas,et al.  TWO-METRIC PROJECTION METHODS FOR CONSTRAINED OPTIMIZATION* , 1984 .

[16]  J. Pang.  On the convergence of a basic iterative method for the implicit complementarity problem , 1982 .

[17]  D. Bertsekas,et al.  Projection methods for variational inequalities with application to the traffic assignment problem , 1982 .

[18]  Wu Li.  Remarks on Convergence of the Matrix Splitting Algorithm for the Symmetric Linear Complementarity Problem , 1993, SIAM J. Optim..

[19]  G. M. Korpelevich.  The extragradient method for finding saddle points and other problems , 1976 .

[20]  Zhi-Quan Luo,et al.  Extension of Hoffman's Error Bound to Polynomial Systems , 1994, SIAM J. Optim..

[21]  Zhi-Quan Luo,et al.  On the rate of convergence of a distributed asynchronous routing algorithm , 1994, IEEE Trans. Autom. Control..

[22]  Paul Tseng,et al.  Error Bound and Reduced-Gradient Projection Algorithms for Convex Minimization over a Polyhedral Set , 1993, SIAM J. Optim..

[23]  Paul Tseng,et al.  On the Rate of Convergence of a Partially Asynchronous Gradient Projection Algorithm , 1991, SIAM J. Optim..

[24]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[25]  O. Mangasarian.  Solution of symmetric linear complementarity problems by iterative methods , 1977 .

[26]  P. Tseng,et al.  On the convergence of the coordinate descent method for convex differentiable minimization , 1992 .

[27]  Paul Tseng,et al.  On the Convergence Rate of Dual Ascent Methods for Linearly Constrained Convex Minimization , 1993, Math. Oper. Res..

[28]  E. Bierstone,et al.  Semianalytic and subanalytic sets , 1988 .

[29]  Clifford Hildreth,et al.  A quadratic programming procedure , 1957 .

[30]  Zhi-Quan Luo,et al.  Error bounds for analytic systems and their applications , 1994, Math. Program..

[31]  Jong-Shi Pang,et al.  A Posteriori Error Bounds for the Linearly-Constrained Variational Inequality Problem , 1987, Math. Oper. Res..

[32]  Marc Teboulle,et al.  Convergence Rate Analysis of Nonquadratic Proximal Methods for Convex and Linear Programming , 1995, Math. Oper. Res..

[33]  A. Goldstein.  Convex programming in Hilbert space , 1964 .

[34]  Paul Tseng,et al.  Partial Proximal Minimization Algorithms for Convex Pprogramming , 1994, SIAM J. Optim..

[35]  J. Dedieu.  Penalty functions in subanalytic optimization , 1992 .

[36]  A. Auslender.  Optimisation : méthodes numériques , 1976 .

[37]  D. D'Esopo,et al.  A convex programming procedure , 1959 .

[38]  P. Tseng.  On linear convergence of iterative methods for the variational inequality problem , 1995 .

[39]  R. Wets,et al.  A Lipschitzian characterization of convex polyhedra , 1969 .

[40]  B. Martinet.  Brève communication. Régularisation d'inéquations variationnelles par approximations successives , 1970 .