Fractional order gradient methods for a general class of convex functions

The aim of this paper is to recall the conventional gradient method and discuss its performance when the target function neither is strongly convex nor has a Lipschitz continuous gradient. By introducing the concepts of fractional order strong convexity and fractional order Lipschitz continuous gradient, some interesting properties of the gradient method are revealed. Furthermore, to keep the asymptotic convergence property of the gradient method when handling such convex functions, the fractional order gradient method is introduced. A careful simulation study is finally provided to demonstrate all the results.

[1]  Jong-Yih Lin,et al.  New IIR filter-based adaptive algorithm in active noise control applications: Commutation error-introduced LMS algorithm and associated convergence assessment by a deterministic approach , 2008, Autom..

[2]  Bin Wang,et al.  A Fast and Robust Level Set Method for Image Segmentation Using Fuzzy Clustering and Lattice Boltzmann Method , 2013, IEEE Transactions on Cybernetics.

[3]  Shu Liang,et al.  Indirect model reference adaptive control for a class of fractional order systems , 2016, Commun. Nonlinear Sci. Numer. Simul..

[4]  Ching-Chang Wong,et al.  A hybrid clustering and gradient descent approach for fuzzy modeling , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[5]  Shaocheng Tong,et al.  Barrier Lyapunov Functions-based adaptive control for a class of nonlinear pure-feedback systems with full state constraints , 2016, Autom..

[6]  Yuquan Chen,et al.  An innovative parameter estimation for fractional-order systems in the presence of outliers , 2017 .

[7]  Farhad Farokhi,et al.  On reconstructability of quadratic utility functions from the iterations in gradient methods , 2015, Autom..

[8]  Yuquan Chen,et al.  Identification for Hammerstein nonlinear ARMAX systems based on multi-innovation fractional order stochastic gradient , 2018, Signal Process..

[9]  Jr. J. Glover.  High Order Algorithms for Adaptive Filters , 1979, IEEE Trans. Commun..

[10]  D. Shanno.  Conditioning of Quasi-Newton Methods for Function Minimization , 1970 .

[11]  William R. Saunders,et al.  Stability and operating constraints of adaptive LMS-based feedback control , 2003, Autom..

[12]  Frank Kretschmer,et al.  An Improved Algorithm for Adaptive Processing , 1978, IEEE Transactions on Aerospace and Electronic Systems.

[13]  Yan Li,et al.  An innovative fractional order LMS based on variable initial value and gradient order , 2017, Signal Process..

[14]  Avinoam Kolodny,et al.  Memristor-Based Multilayer Neural Networks With Online Gradient Descent Training , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[15]  Marco Tulio Angulo,et al.  Nonlinear extremum seeking inspired on second order sliding modes , 2015, Autom..

[16]  Kok Lay Teo,et al.  Parameter estimation for nonlinear time-delay systems with noisy output measurements , 2015, Autom..

[17]  Yuquan Chen,et al.  Study on fractional order gradient methods , 2017, Appl. Math. Comput..