A Modified Spectral Gradient Projection Method for Solving Non-Linear Monotone Equations With Convex Constraints and Its Application

In this paper, we propose a derivative free algorithm for solving non-linear monotone equations with convex constraints. The proposed algorithm combines the method of spectral gradient and the projection method. We also modify the backtracking line search technique. The global convergence of the proposed method is guaranteed, under the mild conditions. Further, the numerical experiments show that the large-scale non-linear equations with convex constraints can be effectively solved with our method. The <inline-formula> <tex-math notation="LaTeX">$L_{1}$ </tex-math></inline-formula>-norm regularized problems in signal reconstruction are studied by using our method.

[1]  Jorge J. Moré,et al.  Computing a Trust Region Step , 1983 .

[2]  José Mario Martínez,et al.  Globally Convergent Inexact Quasi-Newton Methods for Solving Nonlinear Systems , 2003, Numerical Algorithms.

[3]  Hassan Mohammad,et al.  A Positive Spectral Gradient-Like Method For Large-Scale Nonlinear Monotone Equations , 2017 .

[4]  Mikhail V. Solodov,et al.  A Globally Convergent Inexact Newton Method for Systems of Monotone Equations , 1998 .

[5]  N. Xiu,et al.  Convergence Properties of Projection and Contraction Methods for Variational Inequality Problems , 2001 .

[6]  Jong-Shi Pang,et al.  Inexact Newton methods for the nonlinear complementarity problem , 1986, Math. Program..

[7]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[8]  Zhong Wan,et al.  A new nonmonotone spectral residual method for nonsmooth nonlinear equations , 2017, J. Comput. Appl. Math..

[9]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[10]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[11]  Hassan Mohammad,et al.  A descent derivative-free algorithm for nonlinear monotone equations with convex constraints , 2020, RAIRO Oper. Res..

[12]  Xiao Guo,et al.  Spectral residual methods with two new non-monotone line searches for large-scale nonlinear systems of equations , 2015, Appl. Math. Comput..

[13]  Jun Wang,et al.  A Complex-Valued Projection Neural Network for Constrained Optimization of Real Functions in Complex Variables , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[14]  William La Cruz.  A spectral algorithm for large-scale systems of nonlinear monotone equations , 2017, Numerical Algorithms.

[15]  Yunhai Xiao,et al.  Spectral gradient projection method for monotone nonlinear equations with convex constraints , 2009 .

[16]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[17]  Youshen Xia,et al.  An Extended Projection Neural Network for Constrained Optimization , 2004, Neural Computation.

[18]  E. H. Zarantonello.  Projections on Convex Sets in Hilbert Space and Spectral Theory: Part I. Projections on Convex Sets: Part II. Spectral Theory , 1971 .

[19]  Jun Wang,et al.  A projection neural network and its application to constrained optimization problems , 2002 .

[20]  Gang Feng,et al.  A new neural network for solving nonlinear projection equations , 2007, Neural Networks.

[21]  Chuanwei Wang,et al.  A projection method for a system of nonlinear monotone equations with convex constraints , 2007, Math. Methods Oper. Res..

[22]  Poom Kumam,et al.  A hybrid conjugate gradient algorithm for constrained monotone equations with application in compressive sensing , 2020, Heliyon.

[23]  Jing Zhang,et al.  An extended nonmonotone line search technique for large-scale unconstrained optimization , 2018, J. Comput. Appl. Math..

[24]  William W. Hager,et al.  A New Conjugate Gradient Method with Guaranteed Descent and an Efficient Line Search , 2005, SIAM J. Optim..

[25]  Yu-Hong Dai,et al.  A family of spectral gradient methods for optimization , 2018, Comput. Optim. Appl..

[26]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[27]  Poom Kumam,et al.  A modified conjugate gradient method for monotone nonlinear equations with convex constraints , 2019, Applied Numerical Mathematics.

[28]  José Mario Martínez,et al.  Spectral residual method without gradient information for solving large-scale nonlinear systems of equations , 2006, Math. Comput..

[29]  Robert D. Nowak,et al.  A bound optimization approach to wavelet-based image deconvolution , 2005, IEEE International Conference on Image Processing 2005.

[31]  Bing Yang,et al.  An Efficient Implementation of Merrill's Method for Sparse or Partially Separable Systems of Nonlinear Equations , 1991, SIAM J. Optim..

[32]  J.K. Liu,et al.  Partially symmetrical derivative-free Liu–Storey projection method for convex constrained equations , 2018, Int. J. Comput. Math..

[33]  Liqun Qi,et al.  A nonsmooth version of Newton's method , 1993, Math. Program..

[34]  Yuming Feng,et al.  A derivative-free iterative method for nonlinear monotone equations with convex constraints , 2018, Numerical Algorithms.

[35]  Jean-Jacques Fuchs,et al.  Multipath time-delay detection and estimation , 1999, IEEE Trans. Signal Process..

[36]  Weijun Zhou,et al.  Spectral gradient projection method for solving nonlinear monotone equations , 2006 .

[37]  Thomas F. Coleman,et al.  An Interior Trust Region Approach for Nonlinear Minimization Subject to Bounds , 1993, SIAM J. Optim..

[38]  Katya Scheinberg,et al.  Practical inexact proximal quasi-Newton method with global complexity analysis , 2013, Mathematical Programming.

[39]  Yang Liu,et al.  An adaptive family of projection methods for constrained monotone nonlinear equations with applications , 2019, Appl. Math. Comput..

[40]  Duan Li,et al.  On Restart Procedures for the Conjugate Gradient Method , 2004, Numerical Algorithms.

[41]  Wotao Yin,et al.  TR 0707 A Fixed-Point Continuation Method for ` 1-Regularized Minimization with Applications to Compressed Sensing , 2007 .

[42]  Hassan Mohammad,et al.  A Modified Fletcher–Reeves Conjugate Gradient Method for Monotone Nonlinear Equations with Some Applications , 2019, Mathematics.

[43]  Wenzhong Guo,et al.  Two Projection Neural Networks With Reduced Model Complexity for Nonlinear Programming , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[44]  Weijun,et al.  LIMITED MEMORY BFGS METHOD FOR NONLINEAR MONOTONE EQUATIONS , 2007 .

[45]  Yunhai Xiao,et al.  A conjugate gradient method to solve convex constrained monotone equations with applications in compressive sensing , 2013 .

[46]  Songchuan Zhang,et al.  Solving Nonlinear Optimization Problems of Real Functions in Complex Variables by Complex-Valued Iterative Methods , 2018, IEEE Transactions on Cybernetics.

[47]  David Zhang,et al.  A Survey of Sparse Representation: Algorithms and Applications , 2015, IEEE Access.

[48]  Marcos Raydan,et al.  Nonmonotone Spectral Methods for Large-Scale Nonlinear Systems , 2003, Optim. Methods Softw..

[49]  José Mario Martínez,et al.  Spectral Projected Gradient Methods: Review and Perspectives , 2014 .

[50]  Weiyi Liu,et al.  A modified spectral conjugate gradient projection method for signal recovery , 2018, Signal, Image and Video Processing.

[51]  San-yang Liu,et al.  Sufficient Descent Conjugate Gradient Methods for Solving Convex Constrained Nonlinear Monotone Equations , 2014 .

[52]  Zhong Wan,et al.  A Modified Spectral PRP Conjugate Gradient Projection Method for Solving Large-Scale Monotone Equations and Its Application in Compressed Sensing , 2019, Mathematical Problems in Engineering.

[53]  Jun Wang,et al.  A Novel Recurrent Neural Network for Solving Nonlinear Optimization Problems With Inequality Constraints , 2008, IEEE Transactions on Neural Networks.

[54]  Jianhua Ma,et al.  Multivariate spectral gradient projection methodfor nonlinear monotone equations with convex constraints , 2012 .

[55]  Miguel Angel Fernández,et al.  A Newton method using exact jacobians for solving fluid-structure coupling , 2005 .

[56]  Li,et al.  Spectral DY-Type Projection Method for Nonlinear Monotone Systems of Equations , 2015 .

[57]  Yunhai Xiao,et al.  Non-smooth equations based method for l 1 -norm problems with applications to compressed sensing , 2011 .

[58]  Jorge J. Moré,et al.  Benchmarking optimization software with performance profiles , 2001, Math. Program..

[59]  Stephen J. Wright,et al.  Sparse Reconstruction by Separable Approximation , 2008, IEEE Transactions on Signal Processing.

[60]  Shengjie Li,et al.  A projection method for convex constrained monotone nonlinear equations with applications , 2015, Comput. Math. Appl..

[61]  Dmitry M. Malioutov,et al.  Homotopy continuation for sparse signal representation , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[62]  Jing Liu,et al.  Two spectral gradient projection methods for constrained equations and their linear convergence rate , 2015 .