Precompact convergence of the nonconvex Primal-Dual Hybrid Gradient algorithm

Abstract The Primal–Dual Hybrid Gradient (PDHG) algorithm is a powerful algorithm used quite frequently in recent years for solving saddle-point optimization problems. The classical application considers convex functions, and it is well studied in literature. In this paper, we consider the convergence of an alternative formulation of the PDHG algorithm in the nonconvex case under the precompact assumption. The proofs are based on the Kurdyka–Ł ojasiewic functions, that cover a wide range of problems. A simple numerical experiment illustrates the convergence properties.

[1]  K. Kurdyka.  On gradients of functions definable in o-minimal structures , 1998 .

[2]  Guoyin Li,et al.  Douglas–Rachford splitting for nonconvex optimization with application to nonconvex feasibility problems , 2014, Math. Program..

[3]  Bingsheng He,et al.  On the Convergence of Primal-Dual Hybrid Gradient Algorithm , 2014, SIAM J. Imaging Sci..

[4]  Gilles Aubert,et al.  Efficient Schemes for Total Variation Minimization Under Constraints in Image Processing , 2009, SIAM J. Sci. Comput..

[5]  Ernö Robert Csetnek,et al.  On the convergence rate of a forward-backward type primal-dual splitting algorithm for convex optimization problems , 2015 .

[6]  Radu Ioan Bot,et al.  An inertial forward-backward-forward primal-dual splitting algorithm for solving monotone inclusion problems , 2014, Numerical Algorithms.

[7]  Mingqiang Zhu,et al.  An Efficient Primal-Dual Hybrid Gradient Algorithm For Total Variation Image Restoration , 2008 .

[8]  Benar Fux Svaiter,et al.  Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods , 2013, Math. Program..

[9]  Michael Möller,et al.  The Primal-Dual Hybrid Gradient Method for Semiconvex Splittings , 2014, SIAM J. Imaging Sci..

[10]  Tony F. Chan,et al.  A General Framework for a Class of First Order Primal-Dual Algorithms for Convex Optimization in Imaging Science , 2010, SIAM J. Imaging Sci..

[11]  Ting Kei Pong,et al.  Peaceman–Rachford splitting for a class of nonconvex optimization problems , 2015, Comput. Optim. Appl..

[12]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[13]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[14]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[15]  B. V. Dean,et al.  Studies in Linear and Non-Linear Programming. , 1959 .

[16]  Marc Teboulle,et al.  Proximal alternating linearized minimization for nonconvex and nonsmooth problems , 2013, Mathematical Programming.

[17]  R. Kellogg.  A nonlinear alternating direction method , 1969 .

[18]  B. Mordukhovich.  Variational analysis and generalized differentiation , 2006 .

[19]  Guoyin Li,et al.  Global Convergence of Splitting Methods for Nonconvex Composite Optimization , 2014, SIAM J. Optim..

[20]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[21]  R. Tyrrell Rockafellar,et al.  Variational Analysis , 1998, Grundlehren der mathematischen Wissenschaften.

[22]  R. Tyrrell Rockafellar,et al.  Convex Analysis , 1970, Princeton Landmarks in Mathematics and Physics.

[23]  Hédy Attouch,et al.  Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Lojasiewicz Inequality , 2008, Math. Oper. Res..

[24]  Radu Ioan Bot,et al.  On the convergence rate improvement of a primal-dual splitting algorithm for solving monotone inclusion problems , 2013, Mathematical Programming.

[25]  H. H. Rachford,et al.  The Numerical Solution of Parabolic and Elliptic Differential Equations , 1955 .

[26]  Hédy Attouch,et al.  On the convergence of the proximal algorithm for nonsmooth functions involving analytic features , 2008, Math. Program..

[27]  Masao Fukushima,et al.  Some Reformulations and Applications of the Alternating Direction Method of Multipliers , 1994 .

[28]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[29]  Radu Ioan Bot,et al.  A Primal-Dual Splitting Algorithm for Finding Zeros of Sums of Maximal Monotone Operators , 2012, SIAM J. Optim..

[30]  R. Glowinski,et al.  Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics , 1987 .

[31]  Stanley Osher,et al.  A Unified Primal-Dual Algorithm Framework Based on Bregman Iteration , 2010, J. Sci. Comput..