Two Improved Conjugate Gradient Methods with Application in Compressive Sensing and Motion Control

To solve the monotone equations with convex constraints, a novel multiparameterized conjugate gradient method (MPCGM) is designed and analyzed. This kind of conjugate gradient method is derivative-free and can be viewed as a modified version of the famous Fletcher–Reeves (FR) conjugate gradient method. Under approximate conditions, we show that the proposed method has global convergence property. Furthermore, we generalize the MPCGM to solve unconstrained optimization problem and offer another novel conjugate gradient method (NCGM), which satisfies the sufficient descent property without any line search. Global convergence of the NCGM is also proved. Finally, we report some numerical results to show the efficiency of two novel methods. Specifically, their practical applications in compressive sensing and motion control of robot manipulator are also investigated.

[1]  Qingna Li,et al.  A class of derivative-free methods for large-scale nonlinear monotone equations , 2011 .

[2]  Min Sun,et al.  New hybrid conjugate gradient projection method for the convex constrained equations , 2016 .

[3]  Jing Liu,et al.  Two spectral gradient projection methods for constrained equations and their linear convergence rate , 2015 .

[4]  E. Polak,et al.  Note sur la convergence de méthodes de directions conjuguées , 1969 .

[5]  William W. Hager,et al.  A New Conjugate Gradient Method with Guaranteed Descent and an Efficient Line Search , 2005, SIAM J. Optim..

[6]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[7]  Min Sun,et al.  The convergence rate of the proximal alternating direction method of multipliers with indefinite proximal regularization , 2017, Journal of inequalities and applications.

[8]  Zhenjun Shi,et al.  A gradient-related algorithm with inexact line searches , 2004 .

[9]  Masao Fukushima,et al.  A Globally and Superlinearly Convergent Gauss-Newton-Based BFGS Method for Symmetric Nonlinear Equations , 1999, SIAM J. Numer. Anal..

[10]  Min Sun,et al.  A new descent memory gradient method and its global convergence , 2011, J. Syst. Sci. Complex..

[11]  Guang-Ren Duan,et al.  A parametric periodic Lyapunov equation with application in semi-global stabilization of discrete-time periodic systems subject to actuator saturation , 2010, Proceedings of the 2010 American Control Conference.

[12]  Wanyou Cheng,et al.  A PRP type method for systems of monotone equations , 2009, Math. Comput. Model..

[13]  Dong-Hui Li,et al.  A globally convergent BFGS method for nonlinear monotone equations without any merit functions , 2008, Math. Comput..

[14]  Yunhai Xiao,et al.  Spectral gradient projection method for monotone nonlinear equations with convex constraints , 2009 .

[15]  Chuanwei Wang,et al.  A projection method for a system of nonlinear monotone equations with convex constraints , 2007, Math. Methods Oper. Res..

[16]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[17]  Ya-Xiang Yuan,et al.  A Nonlinear Conjugate Gradient Method with a Strong Global Convergence Property , 1999, SIAM J. Optim..

[18]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[19]  Maoying Tian,et al.  Discrete-Time Zhang Neural Networks for Time-Varying Nonlinear Optimization , 2019, Discrete Dynamics in Nature and Society.

[20]  Weijun Zhou,et al.  A descent modified Polak–Ribière–Polyak conjugate gradient method and its global convergence , 2006 .

[21]  Min Sun,et al.  General Five-Step Discrete-Time Zhang Neural Network for Time-Varying Nonlinear Optimization , 2019, Bulletin of the Malaysian Mathematical Sciences Society.

[22]  K. Toh,et al.  Superlinear Convergence of a Newton-Type Algorithm for Monotone Equations , 2005 .

[23]  A prediction-correction primal-dual hybrid gradient method for convex programming with linear constraints , 2018 .

[24]  J. J. Moré,et al.  A Characterization of Superlinear Convergence and its Application to Quasi-Newton Methods , 1973 .

[25]  Marcos Raydan,et al.  Nonmonotone Spectral Methods for Large-Scale Nonlinear Systems , 2003, Optim. Methods Softw..

[26]  C. Storey,et al.  Efficient generalized conjugate gradient algorithms, part 1: Theory , 1991 .

[27]  Yunhai Xiao,et al.  A conjugate gradient method to solve convex constrained monotone equations with applications in compressive sensing , 2013 .

[28]  Weijun Zhou,et al.  Spectral gradient projection method for solving nonlinear monotone equations , 2006 .

[29]  Li Zhang,et al.  Global convergence of a modified Fletcher–Reeves conjugate gradient method with Armijo-type line search , 2006, Numerische Mathematik.

[30]  Min Sun,et al.  Iterative Algorithms for Symmetric Positive Semidefinite Solutions of the Lyapunov Matrix Equations , 2020 .

[31]  Yang Shi,et al.  General four-step discrete-time zeroing and derivative dynamics applied to time-varying nonlinear optimization , 2019, J. Comput. Appl. Math..

[32]  José Mario Martínez,et al.  Spectral residual method without gradient information for solving large-scale nonlinear systems of equations , 2006, Math. Comput..