Accelerating the DC algorithm for smooth functions

We introduce two new algorithms to minimise smooth difference of convex (DC) functions that accelerate the convergence of the classical DC algorithm (DCA). We prove that the point computed by DCA can be used to define a descent direction for the objective function evaluated at this point. Our algorithms are based on a combination of DCA together with a line search step that uses this descent direction. Convergence of the algorithms is proved and the rate of convergence is analysed under the Łojasiewicz property of the objective function. We apply our algorithms to a class of smooth DC programs arising in the study of biochemical reaction networks, where the objective function is real analytic and thus satisfies the Łojasiewicz property. Numerical tests on various biochemical models clearly show that our algorithms outperform DCA, being on average more than four times faster in both computational time and the number of iterations. Numerical experiments show that the algorithms are globally convergent to a non-equilibrium steady state of various biochemical networks, with only chemically consistent restrictions on the network topology.

[1]  H. Ngai,et al.  Convergence Analysis of DC Algorithm for DC programming with subanalytic data , 2010 .

[2]  Michael A. Saunders,et al.  Proximal Newton-Type Methods for Minimizing Composite Functions , 2012, SIAM J. Optim..

[3]  Moudafi,et al.  ON THE CONVERGENCE OF AN APPROXIMATE PROXIMAL METHOD FOR DC FUNCTIONS , 2006 .

[4]  S. Łojasiewicz Ensembles semi-analytiques , 1965 .

[5]  Ronan M. T. Fleming,et al.  A community-driven global reconstruction of human metabolism , 2013, Nature Biotechnology.

[6]  Hédy Attouch,et al.  On the convergence of the proximal algorithm for nonsmooth functions involving analytic features , 2008, Math. Program..

[7]  Le Thi Hoai An,et al.  A D.C. Optimization Algorithm for Solving the Trust-Region Subproblem , 1998, SIAM J. Optim..

[8]  Jason Weston,et al.  Trading convexity for scalability , 2006, ICML.

[9]  Yurii Nesterov,et al.  Gradient methods for minimizing composite functions , 2012, Mathematical Programming.

[10]  Le Thi Hoai An,et al.  On solving Linear Complementarity Problems by DC programming and DCA , 2011, Comput. Optim. Appl..

[11]  Stephen J. Wright,et al.  Numerical Optimization (Springer Series in Operations Research and Financial Engineering) , 2000 .

[12]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[13]  Adrian S. Lewis,et al.  The [barred L]ojasiewicz Inequality for Nonsmooth Subanalytic Functions with Applications to Subgradient Dynamical Systems , 2006, SIAM J. Optim..

[14]  M. Fukushima,et al.  A minimization method for the sum of a convex function and a continuously differentiable function , 1981 .

[15]  Le Thi Hoai An,et al.  The DC (Difference of Convex Functions) Programming and DCA Revisited with DC Models of Real World Nonconvex Optimization Problems , 2005, Ann. Oper. Res..

[16]  Robert E. Mahony,et al.  Convergence of the Iterates of Descent Methods for Analytic Cost Functions , 2005, SIAM J. Optim..

[17]  Ronan M. T. Fleming,et al.  Globally convergent algorithms for finding zeros of duplomonotone mappings , 2015, Optim. Lett..

[18]  Marc Teboulle,et al.  Proximal alternating linearized minimization for nonconvex and nonsmooth problems , 2013, Mathematical Programming.

[19]  El Bernoussi Souad,et al.  Algorithms for Solving a Class of Nonconvex Optimization Problems. Methods of Subgradients , 1986 .

[20]  Harold R. Parks,et al.  A Primer of Real Analytic Functions , 1992 .

[21]  Christoph Schnörr,et al.  Variational Reconstruction with DC-Programming , 2007 .

[22]  M. Fukushima,et al.  A generalized proximal point algorithm for certain non-convex minimization problems , 1981 .

[23]  Le Thi Hoai An,et al.  Numerical solution for optimization over the efficient set by d.c. optimization algorithms , 1996, Oper. Res. Lett..

[24]  Hongwei Liu,et al.  A Barzilai–Borwein type method for stochastic linear complementarity problems , 2013, Numerical Algorithms.

[25]  Guoyin Li,et al.  Douglas–Rachford splitting for nonconvex optimization with application to nonconvex feasibility problems , 2014, Math. Program..

[26]  W. Gander,et al.  A D.C. OPTIMIZATION ALGORITHM FOR SOLVING THE TRUST-REGION SUBPROBLEM∗ , 1998 .

[27]  David A. Fell,et al.  Detection of stoichiometric inconsistencies in biomolecular models , 2008, Bioinform..