A New Three-Term Hestenes-Stiefel Type Method for Nonlinear Monotone Operator Equations and Image Restoration

In this article, a derivative-free method of Hestenes-Stiefel type is proposed for solving system of monotone operator equations with convex constraints. The method proposed is matrix-free, and its sequence of search directions are bounded and satisfies the sufficient descent condition. The global convergence of the proposed approach is established under the assumptions that the underlying operator is monotone and Lipschitz continuous. Numerical experiment results are reported to show the efficiency of the proposed method. Furthermore, to illustrate the applicability of the proposed method, it is used in restoring blurred images.

[1]  Abdulkarim Hassan Ibrahim,et al.  Derivative-free HS-DY-type method for solving nonlinear equations and image restoration , 2020, Heliyon.

[2]  Marcos Raydan,et al.  Nonmonotone Spectral Methods for Large-Scale Nonlinear Systems , 2003, Optim. Methods Softw..

[3]  Boris Polyak.  The conjugate gradient method in extremal problems , 1969 .

[4]  Poom Kumam,et al.  An efficient gradient-free projection algorithm for constrained nonlinear equations and image restoration , 2021 .

[5]  Poom Kumam,et al.  A hybrid conjugate gradient algorithm for constrained monotone equations with application in compressive sensing , 2020, Heliyon.

[6]  Poom Kumam,et al.  Least-Square-Based Three-Term Conjugate Gradient Projection Method for ℓ1-Norm Problems with Application to Compressed Sensing , 2020, Mathematics.

[7]  Yuming Feng,et al.  A derivative-free iterative method for nonlinear monotone equations with convex constraints , 2018, Numerical Algorithms.

[8]  M. Mamat,et al.  A new family of conjugate gradient methods for unconstrained optimization , 2011, 2011 Fourth International Conference on Modeling, Simulation and Applied Optimization.

[9]  Hassan Mohammad,et al.  A note on the spectral gradient projection method for nonlinear monotone equations with applications , 2020, Comput. Appl. Math..

[10]  P. Kaelo,et al.  A globally convergent projection method for a system of nonlinear monotone equations , 2020, Int. J. Comput. Math..

[11]  Li Zheng,et al.  A Modified Spectral Gradient Projection Method for Solving Non-Linear Monotone Equations With Convex Constraints and Its Application , 2020, IEEE Access.

[12]  Yaping Hu,et al.  Wei–Yao–Liu conjugate gradient projection algorithm for nonlinear monotone equations with convex constraints , 2015, Int. J. Comput. Math..

[13]  Wotao Yin,et al.  TR 0707 A Fixed-Point Continuation Method for ` 1-Regularized Minimization with Applications to Compressed Sensing , 2007 .

[14]  José Mario Martínez,et al.  Practical quasi-Newton methods for solving nonlinear systems , 2000 .

[15]  Wanyou Cheng,et al.  A PRP type method for systems of monotone equations , 2009, Math. Comput. Model..

[16]  Yunhai Xiao,et al.  Non-smooth equations based method for l 1 -norm problems with applications to compressed sensing , 2011 .

[17]  Weijun Zhou,et al.  Spectral gradient projection method for solving nonlinear monotone equations , 2006 .

[18]  Dong-Hui Li,et al.  A globally convergent BFGS method for nonlinear monotone equations without any merit functions , 2008, Math. Comput..

[19]  Bing Zheng,et al.  Two New Dai–Liao-Type Conjugate Gradient Methods for Unconstrained Optimization Problems , 2017, J. Optim. Theory Appl..

[20]  D. Donoho.  For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solution , 2006 .

[21]  M. Waziri,et al.  Two optimal Hager-Zhang conjugate gradient methods for solving monotone nonlinear equations , 2020 .

[22]  Jin-Bao Jian,et al.  A New Conjugate Gradient Projection Method for Convex Constrained Nonlinear Equations , 2020, Complex..

[23]  Yunhai Xiao,et al.  Spectral gradient projection method for monotone nonlinear equations with convex constraints , 2009 .

[24]  Masao Fukushima,et al.  A Globally and Superlinearly Convergent Gauss-Newton-Based BFGS Method for Symmetric Nonlinear Equations , 1999, SIAM J. Numer. Anal..

[25]  Seyed Mehdi Lajevardi,et al.  Structural similarity classifier for facial expression recognition , 2014, Signal Image Video Process..

[26]  Jorge J. Moré,et al.  Benchmarking optimization software with performance profiles , 2001, Math. Program..

[27]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[28]  E. Polak,et al.  Note sur la convergence de méthodes de directions conjuguées , 1969 .

[29]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[30]  Poom Kumam,et al.  A Family of Derivative-Free Conjugate Gradient Methods for Constrained Nonlinear Equations and Image Restoration , 2020, IEEE Access.

[31]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[32]  Li,et al.  Spectral DY-Type Projection Method for Nonlinear Monotone Systems of Equations , 2015 .

[33]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[34]  Jing Liu,et al.  Two Improved Conjugate Gradient Methods with Application in Compressive Sensing and Motion Control , 2020 .

[35]  Yunhai Xiao,et al.  A class of conjugate gradient methods for convex constrained monotone equations , 2017 .

[36]  J. J. Moré,et al.  A Characterization of Superlinear Convergence and its Application to Quasi-Newton Methods , 1973 .

[37]  A. Morgan,et al.  A methodology for solving chemical equilibrium systems , 1987 .

[38]  Chuanjiang He,et al.  An efficient three-term conjugate gradient method for nonlinear monotone equations with convex constraints , 2018, Calcolo.

[39]  José Mario Martínez,et al.  Spectral residual method without gradient information for solving large-scale nonlinear systems of equations , 2006, Math. Comput..

[40]  Bing Yang,et al.  An Efficient Implementation of Merrill's Method for Sparse or Partially Separable Systems of Nonlinear Equations , 1991, SIAM J. Optim..

[41]  Li Zheng,et al.  A conjugate gradient projection method for solving equations with convex constraints , 2020, J. Comput. Appl. Math..

[43]  Poom Kumam,et al.  A descent Dai-Liao conjugate gradient method for nonlinear equations , 2018, Numerical Algorithms.

[44]  Allen J. Wood,et al.  Power Generation, Operation, and Control , 1984 .

[45]  Jerry D. Gibson,et al.  Handbook of Image and Video Processing , 2000 .

[46]  K. Toh,et al.  Superlinear Convergence of a Newton-Type Algorithm for Monotone Equations , 2005 .

[47]  Zhi-feng Dai,et al.  A Modified Hestenes-Stiefel-Type Derivative-Free Method for Large-Scale Nonlinear Monotone Equations , 2020, Mathematics.

[48]  Qingna Li,et al.  A class of derivative-free methods for large-scale nonlinear monotone equations , 2011 .

[49]  S. Djordjevic.  New Hybrid Conjugate Gradient Method As A Convex Combination of Ls and Fr Methods , 2019, Acta Mathematica Scientia.

[50]  Mingyang Sun,et al.  A Class of Derivative-Free CG Projection Methods for Nonsmooth Equations with an Application to the LASSO Problem , 2020, Bulletin of the Iranian Mathematical Society.

[51]  Zhifeng Dai,et al.  New technical indicators and stock returns predictability , 2021 .

[52]  Mikhail V. Solodov,et al.  A Globally Convergent Inexact Newton Method for Systems of Monotone Equations , 1998 .

[53]  Yunhai Xiao,et al.  A conjugate gradient method to solve convex constrained monotone equations with applications in compressive sensing , 2013 .

[54]  Z. Salleh,et al.  A New Modified Three-Term Hestenes–Stiefel Conjugate Gradient Method with Sufficient Descent Property and Its Global Convergence , 2018, Journal of Optimization.

[55]  M. Waziri,et al.  Improved conjugate gradient method for nonlinear system of equations , 2020, Comput. Appl. Math..

[56]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[57]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .