A modified conjugate gradient method for monotone nonlinear equations with convex constraints

Abstract In this paper, a modified Hestenes-Stiefel (HS) spectral conjugate gradient (CG) method for monotone nonlinear equations with convex constraints is proposed based on projection technique. The method can be viewed as an extension of a modified HS-CG method for unconstrained optimization proposed by Amini et al. (Optimization Methods and Software, pp: 1-13, 2018). A new search direction is obtained by incorporating the idea of spectral gradient parameter and some modification of the conjugate gradient parameter. The proposed method is derivative-free and requires low memory which makes it suitable for large scale monotone nonlinear equations. Global convergence of the method is established under suitable assumptions. Preliminary numerical comparisons with some existing methods are given to show the efficiency of our proposed method. Furthermore, the proposed method is successfully applied to solve sparse signal reconstruction in compressive sensing.

[1]  Yongtang Shi,et al.  Two New PRP Conjugate Gradient Algorithms for Minimization Optimization Models , 2015, PloS one.

[2]  Weijun Zhou,et al.  An Inexact PRP Conjugate Gradient Method for Symmetric Nonlinear Equations , 2014 .

[3]  Yuming Feng,et al.  A derivative-free iterative method for nonlinear monotone equations with convex constraints , 2018, Numerical Algorithms.

[4]  Na Yuan.  A derivative-free projection method for solving convex constrained monotone equations , 2017 .

[5]  Hassan Mohammad,et al.  A Positive Spectral Gradient-Like Method For Large-Scale Nonlinear Monotone Equations , 2017 .

[6]  M. Fukushima,et al.  On the Rate of Convergence of the Levenberg-Marquardt Method , 2001 .

[7]  Yunhai Xiao,et al.  Spectral gradient projection method for monotone nonlinear equations with convex constraints , 2009 .

[8]  Mikhail V. Solodov,et al.  A Globally Convergent Inexact Newton Method for Systems of Monotone Equations , 1998 .

[9]  M. V. Solodovy,et al.  Newton-type Methods with Generalized Distances For Constrained Optimization , 1997 .

[10]  Yunhai Xiao,et al.  A conjugate gradient method to solve convex constrained monotone equations with applications in compressive sensing , 2013 .

[11]  Maojun Zhang,et al.  A three-terms Polak-Ribière-Polyak conjugate gradient algorithm for large-scale nonlinear equations , 2015, J. Comput. Appl. Math..

[12]  C. G. Broyden.  A Class of Methods for Solving Nonlinear Simultaneous Equations , 1965 .

[13]  Shengjie Li,et al.  A projection method for convex constrained monotone nonlinear equations with applications , 2015, Comput. Math. Appl..

[14]  Weijun Zhou,et al.  Spectral gradient projection method for solving nonlinear monotone equations , 2006 .

[15]  William W. Hager,et al.  Algorithm 851: CG_DESCENT, a conjugate gradient method with guaranteed descent , 2006, TOMS.

[16]  Chuanwei Wang,et al.  A superlinearly convergent projection method for constrained systems of nonlinear equations , 2009, J. Glob. Optim..

[17]  Yunhai Xiao,et al.  Non-smooth equations based method for l 1 -norm problems with applications to compressed sensing , 2011 .

[18]  Yong Li,et al.  A Modified Hestenes and Stiefel Conjugate Gradient Algorithm for Large-Scale Nonsmooth Minimizations and Nonlinear Equations , 2015, Journal of Optimization Theory and Applications.

[19]  Benedetta Morini,et al.  Quasi-Newton methods for constrained nonlinear systems: complexity analysis and applications , 2018, Comput. Optim. Appl..

[20]  Poom Kumam,et al.  A descent Dai-Liao conjugate gradient method for nonlinear equations , 2018, Numerical Algorithms.

[21]  Poom Kumam,et al.  A New Hybrid Spectral Gradient Projection Method for Monotone System of Nonlinear Equations with Convex Constraints , 2018 .

[22]  Wah June Leong,et al.  A New Newton's Method with Diagonal Jacobian Approximation for Systems of Nonlinear Equations , 2010 .

[23]  Wah June Leong,et al.  A matrix-free quasi-Newton method for solving large-scale nonlinear systems , 2011, Comput. Math. Appl..

[24]  Jorge J. Moré,et al.  Benchmarking optimization software with performance profiles , 2001, Math. Program..

[25]  Jong-Shi Pang,et al.  Inexact Newton methods for the nonlinear complementarity problem , 1986, Math. Program..

[26]  Duan Li,et al.  Monotonicity of Fixed Point and Normal Mappings Associated with Variational Inequality and Its Application , 2000, SIAM J. Optim..

[27]  Keyvan Amini,et al.  A modified Hestenes–Stiefel conjugate gradient method with an optimal property , 2019, Optim. Methods Softw..

[28]  C. Kelley.  Iterative Methods for Linear and Nonlinear Equations , 1987 .

[29]  Jianhua Ma,et al.  Multivariate spectral gradient projection methodfor nonlinear monotone equations with convex constraints , 2012 .

[30]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[31]  Poom Kumam,et al.  A Descent Dai-Liao Projection Method for Convex Constrained Nonlinear Monotone Equations with Applications , 2018 .

[32]  Chuanwei Wang,et al.  A projection method for a system of nonlinear monotone equations with convex constraints , 2007, Math. Methods Oper. Res..

[33]  Christian Kanzow,et al.  An interior-point affine-scaling trust-region method for semismooth equations with box constraints , 2007, Comput. Optim. Appl..

[34]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[35]  K. Toh,et al.  Superlinear Convergence of a Newton-Type Algorithm for Monotone Equations , 2005 .

[36]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[37]  Stefania Bellavia,et al.  Subspace Trust-Region Methods for Large Bound-Constrained Nonlinear Equations , 2006, SIAM J. Numer. Anal..

[38]  William La Cruz.  A spectral algorithm for large-scale systems of nonlinear monotone equations , 2017, Numerical Algorithms.