Local convergence of the Levenberg–Marquardt method under Hölder metric subregularity

We describe and analyse Levenberg–Marquardt methods for solving systems of nonlinear equations. More specifically, we propose an adaptive formula for the Levenberg–Marquardt parameter and analyse the local convergence of the method under Hölder metric subregularity of the function defining the equation and Hölder continuity of its gradient mapping. Further, we analyse the local convergence of the method under the additional assumption that the Łojasiewicz gradient inequality holds. We finally report encouraging numerical results confirming the theoretical findings for the problem of computing moiety conserved steady states in biochemical reaction networks. This problem can be cast as finding a solution of a system of nonlinear equations, where the associated mapping satisfies the Łojasiewicz gradient inequality assumption.

[1]  Alexander Y. Kruger,et al.  Error Bounds and Hölder Metric Subregularity , 2014, 1411.6414.

[2]  D. Klatte Book review: Implicit Functions and Solution Mappings:A View from Variational Analysis. Second Edition. By A. L. Dontchev and R. T. Rockafellar. Springer, New York, 2014 , 2015 .

[3]  Jorge J. Moré,et al.  Benchmarking optimization software with performance profiles , 2001, Math. Program..

[4]  M. Fukushima,et al.  On the Rate of Convergence of the Levenberg-Marquardt Method , 2001 .

[5]  B. Mordukhovich Variational analysis and generalized differentiation , 2006 .

[6]  Andreas Fischer,et al.  Local behavior of an iterative framework for generalized equations with nonisolated solutions , 2002, Math. Program..

[7]  Benar Fux Svaiter,et al.  Algebraic rules for computing the regularization parameter of the Levenberg–Marquardt method , 2016, Comput. Optim. Appl..

[8]  A. Hasegawa Plasma instabilities and nonlinear effects , 1975 .

[9]  Jorge J. Moré,et al.  Testing Unconstrained Optimization Software , 1981, TOMS.

[10]  Hédy Attouch,et al.  On the convergence of the proximal algorithm for nonsmooth functions involving analytic features , 2008, Math. Program..

[11]  Boris S. Mordukhovich,et al.  Hölder Metric Subregularity with Applications to Proximal Point Method , 2012, SIAM J. Optim..

[12]  Harold R. Parks,et al.  A Primer of Real Analytic Functions , 1992 .

[13]  David A. Fell,et al.  Detection of stoichiometric inconsistencies in biomolecular models , 2008, Bioinform..

[14]  Ronan M. T. Fleming,et al.  Conditions for duality between fluxes and concentrations in biochemical networks. , 2015, Journal of theoretical biology.

[15]  Lei Guo,et al.  Solving Mathematical Programs with Equilibrium Constraints , 2015, J. Optim. Theory Appl..

[16]  Jong-Shi Pang,et al.  Error bounds in mathematical programming , 1997, Math. Program..

[17]  Ya-Xiang Yuan,et al.  On the Quadratic Convergence of the Levenberg-Marquardt Method without Nonsingularity Assumption , 2005, Computing.

[18]  Jinyan Fan,et al.  The modified Levenberg-Marquardt method for nonlinear equations with cubic convergence , 2012, Math. Comput..

[19]  Alexey F. Izmailov,et al.  Error bounds for 2-regular mappings with Lipschitzian derivatives and their applications , 2001, Math. Program..

[20]  R. Rockafellar,et al.  Implicit Functions and Solution Mappings , 2009 .

[21]  Stefania Bellavia,et al.  Strong local convergence properties of adaptive regularized methods for nonlinear least squares , 2015 .

[22]  Krzysztof Kurdyka,et al.  Separation of real algebraic sets and the Łojasiewicz exponent , 2014 .

[23]  Carl Tim Kelley,et al.  Iterative methods for optimization , 1999, Frontiers in applied mathematics.

[24]  M. Fukushima,et al.  Erratum to Levenberg-Marquardt methods with strong local convergence properties for solving nonlinear equations with convex constraints , 2005 .

[25]  Ronan M. T. Fleming,et al.  Globally convergent algorithms for finding zeros of duplomonotone mappings , 2015, Optim. Lett..

[26]  Jinyan Fan,et al.  A note on the Levenberg-Marquardt parameter , 2009, Appl. Math. Comput..

[27]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[28]  A. F. Izmailov,et al.  Newton-Type Methods for Optimization and Variational Problems , 2014 .

[29]  Lihua Jiang,et al.  Some research on Levenberg-Marquardt method for the nonlinear equations , 2007, Appl. Math. Comput..

[30]  Francisco J. Planes,et al.  Creation and analysis of biochemical constraint-based models: the COBRA Toolbox v3.0. , 2017, 1710.04038.

[31]  Alfredo N. Iusem,et al.  The effect of calmness on the solution set of systems of nonlinear equations , 2013, Math. Program..

[32]  Huynh van Ngai,et al.  Global Error Bounds for Systems of Convex Polynomials over Polyhedral Constraints , 2015, SIAM J. Optim..

[33]  Andreas Fischer,et al.  A Globally Convergent LP-Newton Method , 2016, SIAM J. Optim..

[34]  Boris S. Mordukhovich,et al.  Higher-order metric subregularity and its applications , 2015, J. Glob. Optim..

[35]  Jinyan Fan Convergence Rate of The Trust Region Method for Nonlinear Equations Under Local Error Bound Condition , 2006, Comput. Optim. Appl..

[36]  Alexander Kruger,et al.  Strong metric subregularity of mappings in variational analysis and optimization , 2017, 1701.02078.

[37]  Ronan M. T. Fleming,et al.  Accelerating the DC algorithm for smooth functions , 2018, Math. Program..

[38]  A. Hoffman On approximate solutions of systems of linear inequalities , 1952 .

[39]  Alexey F. Izmailov,et al.  The Theory of 2-Regularity for Mappings with Lipschitzian Deriatives and its Applications to Optimality Conditions , 2002, Math. Oper. Res..

[40]  Hà Huy Vui Global Hölderian Error Bound for Nondegenerate Polynomials , 2013, SIAM J. Optim..

[41]  J. Bolte,et al.  Characterizations of Lojasiewicz inequalities: Subgradient flows, talweg, convexity , 2009 .

[42]  Gui-Hua Lin,et al.  Improved convergence results for a modified Levenberg–Marquardt method for nonlinear equations and applications in MPCC , 2016, Optim. Methods Softw..

[43]  G. Whitham,et al.  Linear and Nonlinear Waves , 1976 .

[44]  Adrian S. Lewis,et al.  The [barred L]ojasiewicz Inequality for Nonsmooth Subanalytic Functions with Applications to Subgradient Dynamical Systems , 2006, SIAM J. Optim..

[45]  J. Gwozdziewicz The Łojasiewicz exponent of an analytic function at an isolated zero , 1999 .

[46]  Steffen Klamt,et al.  Hypergraphs and Cellular Networks , 2009, PLoS Comput. Biol..

[47]  Anne Richelle,et al.  Creation and analysis of biochemical constraint-based models using the COBRA Toolbox v.3.0 , 2019, Nature Protocols.

[48]  Ronan M. T. Fleming,et al.  Identification of Conserved Moieties in Metabolic Networks by Graph Theoretical Analysis of Atom Transition Networks , 2016, PLoS Comput. Biol..

[49]  Nicholas I. M. Gould,et al.  Convergence of a Regularized Euclidean Residual Algorithm for Nonlinear Least-Squares , 2010, SIAM J. Numer. Anal..

[50]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[51]  G. Eilenberger Solitons: Mathematical Methods for Physicists , 1983 .

[52]  C. Kanzow Levenberg-Marquardt methods for constrained nonlinear equations with strong local convergence properties , 2004 .

[53]  Ya-Xiang Yuan,et al.  Recent advances in trust region algorithms , 2015, Mathematical Programming.

[54]  Ronan M. T. Fleming,et al.  Mass conserved elementary kinetics is sufficient for the existence of a non-equilibrium steady state concentration. , 2011, Journal of theoretical biology.

[55]  S. Łojasiewicz Ensembles semi-analytiques , 1965 .

[56]  Benar Fux Svaiter,et al.  Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods , 2013, Math. Program..