A general double-proximal gradient algorithm for d.c. programming

The possibilities of exploiting the special structure of d.c. programs, which consist of optimising the difference of convex functions, are currently more or less limited to variants of the DCA proposed by Pham Dinh Tao and Le Thi Hoai An in 1997. These assume that either the convex or the concave part, or both, are evaluated by one of their subgradients. In this paper we propose an algorithm which allows the evaluation of both the concave and the convex part by their proximal points. Additionally, we allow a smooth part, which is evaluated via its gradient. In the spirit of primal-dual splitting algorithms, the concave part might be the composition of a concave function with a linear operator, which are, however, evaluated separately. For this algorithm we show that every cluster point is a solution of the optimisation problem. Furthermore, we show the connection to the Toland dual problem and prove a descent property for the objective function values of a primal-dual formulation of the problem. Convergence of the iterates is shown if this objective function satisfies the Kurdyka–Łojasiewicz property. In the last part, we apply the algorithm to an image processing model.

[1]  T. P. Dinh,et al.  Convex analysis approach to d.c. programming: Theory, Algorithm and Applications , 1997 .

[2]  J. Toland.  A duality principle for non-convex optimisation and the calculus of variations , 1979 .

[3]  Liguo Jiao,et al.  Convergence Analysis of Algorithms for DC Programming , 2015, 1508.03899.

[4]  Paul-Emile Maingé,et al.  Convergence of New Inertial Proximal Methods for DC Programming , 2008, SIAM J. Optim..

[5]  Jong-Shi Pang,et al.  Computing B-Stationary Points of Nonsmooth DC Programs , 2015, Math. Oper. Res..

[6]  Hédy Attouch,et al.  Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Lojasiewicz Inequality , 2008, Math. Oper. Res..

[7]  Boris Polyak.  Some methods of speeding up the convergence of iteration methods , 1964 .

[8]  Tong Zhang.  Some sharp performance bounds for least squares regression with L1 regularization , 2009, 0908.2869.

[9]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[10]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[11]  Radu Ioan Bot,et al.  An inertial forward–backward algorithm for the minimization of the sum of two nonconvex functions , 2014, EURO J. Comput. Optim..

[12]  Hédy Attouch,et al.  On the convergence of the proximal algorithm for nonsmooth functions involving analytic features , 2008, Math. Program..

[13]  C. Zălinescu.  Convex analysis in general vector spaces , 2002 .

[14]  Nguyen Thai An,et al.  Convergence analysis of a proximal point algorithm for minimizing differences of functions , 2015, 1504.08079.

[15]  Benar Fux Svaiter,et al.  Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods , 2013, Math. Program..

[16]  Ronan M. T. Fleming,et al.  Accelerating the DC algorithm for smooth functions , 2018, Math. Program..

[17]  Stéphane Canu,et al.  Recovering Sparse Signals With a Certain Family of Nonconvex Penalties and DC Programming , 2009, IEEE Transactions on Signal Processing.

[18]  Marc Teboulle,et al.  A fast dual proximal gradient algorithm for convex minimization and applications , 2014, Oper. Res. Lett..

[19]  Fariborz Maseeh,et al.  Convergence Analysis of a Proximal Point Algorithm for Minimizing Differences of Functions , 2017 .

[20]  Jack Xin,et al.  A Weighted Difference of Anisotropic and Isotropic Total Variation Model for Image Processing , 2015, SIAM J. Imaging Sci..

[21]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[22]  Le Thi Hoai An,et al.  DCA based algorithms for feature selection in multi-class support vector machine , 2017, Ann. Oper. Res..

[23]  R. Horst,et al.  DC Programming: Overview , 1999 .

[24]  K. Kurdyka.  On gradients of functions definable in o-minimal structures , 1998 .

[25]  J. Toland.  Duality in nonconvex optimization , 1978 .

[26]  Radu Ioan Bot,et al.  Convergence Analysis for a Primal-Dual Monotone + Skew Splitting Algorithm with Applications to Total Variation Minimization , 2012, Journal of Mathematical Imaging and Vision.

[27]  Thomas Brox,et al.  iPiano: Inertial Proximal Algorithm for Nonconvex Optimization , 2014, SIAM J. Imaging Sci..

[28]  Marc Teboulle,et al.  Proximal alternating linearized minimization for nonconvex and nonsmooth problems , 2013, Mathematical Programming.

[29]  Bastian Goldlücke,et al.  Variational Analysis , 2014, Computer Vision, A Reference Guide.

[30]  Wen-yuSun,et al.  PROXIMAL POINT ALGORITHM FOR MINIMIZATION OF DC FUNCTION , 2003 .

[31]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.