An Optimal PID Control Algorithm for Training Feedforward Neural Networks

The training problem of feedforward neural networks (FNNs) is formulated into a proportional integral and derivative (PID) control problem of a linear discrete dynamic system in terms of the estimation error. The robust control approach greatly facilitates the analysis and design of robust learning algorithms for multiple-input-multiple-output (MIMO) FNNs using robust control methods. The drawbacks of some existing learning algorithms can therefore be revealed clearly, and an optimal robust PID-learning algorithm is developed. The optimal learning parameters can be found by utilizing linear matrix inequality optimization techniques. Theoretical analysis and examples including function approximation, system identification, exclusive-or (XOR) and encoder problems are provided to illustrate the results.

[1]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[2]  C. Charalambous,et al.  Conjugate gradient algorithm for efficient training of artifi-cial neural networks , 1990 .

[3]  Jooyoung Park,et al.  Universal Approximation Using Radial-Basis-Function Networks , 1991, Neural Computation.

[4]  Hideaki Sakai,et al.  A real-time learning algorithm for a multilayered neural network based on the extended Kalman filter , 1992, IEEE Trans. Signal Process..

[5]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[6]  P. S. Sastry,et al.  Analysis of the back-propagation algorithm with momentum , 1994, IEEE Trans. Neural Networks.

[7]  Dilip Sarkar,et al.  Methods to speed up error back-propagation learning algorithm , 1995, CSUR.

[8]  P. Sunthar,et al.  The generalized proportional-integral-derivative (PID) gradient descent back propagation algorithm , 1995, Neural Networks.

[9]  Martin T. Hagan,et al.  Neural network design , 1995 .

[10]  Stanislaw Osowski,et al.  Fast Second Order Learning Algorithm for Feedforward Multilayer Neural Networks and its Applications , 1996, Neural Networks.

[11]  Ah Chung Tsoi,et al.  Universal Approximation Using Feedforward Neural Networks: A Survey of Some Existing Methods, and Some New Results , 1998, Neural Networks.

[12]  Leszek Rutkowski,et al.  A fast training algorithm for neural networks , 1998 .

[13]  Hugang Han,et al.  Adaptive control of a class of nonlinear systems with nonlinearly parameterized fuzzy approximators , 2001, IEEE Trans. Fuzzy Syst..

[14]  Kiyoshi Nishiyama,et al.  H∞-learning of layered neural networks , 2001, IEEE Trans. Neural Networks.

[15]  Tao Zhang,et al.  Stable Adaptive Neural Network Control , 2001, The Springer International Series on Asian Studies in Computer and Information Science.

[16]  Miguel Pinzolas,et al.  Neighborhood based Levenberg-Marquardt algorithm for neural network training , 2002, IEEE Trans. Neural Networks.

[17]  Zhihong Man,et al.  A New Adaptive Backpropagation Algorithm Based on Lyapunov Stability Theory for Neural Networks , 2006, IEEE Transactions on Neural Networks.

[18]  Laxmidhar Behera,et al.  On Adaptive Learning Rate That Guarantees Convergence in Feedforward Networks , 2006, IEEE Transactions on Neural Networks.

[19]  Wei Wu,et al.  Convergence of BP Algorithm with Variable Learning Rates for FNN Training , 2006, 2006 Fifth Mexican International Conference on Artificial Intelligence.

[20]  Stephen A. Billings,et al.  Model structure selection using an integrated forward orthogonal search algorithm assisted by squared correlation and mutual information , 2008, Int. J. Model. Identif. Control..

[21]  Shuang Cong,et al.  PID-Like Neural Network Nonlinear Adaptive Control for Uncertain Multivariable Motion Control Systems , 2009, IEEE Transactions on Industrial Electronics.

[22]  Jang-Hyun Park,et al.  Adaptive Neural Control for Strict-Feedback Nonlinear Systems Without Backstepping , 2009, IEEE Transactions on Neural Networks.

[23]  Okyay Kaynak,et al.  Type 2 Fuzzy Neural Structure for Identification and Control of Time-Varying Plants , 2010, IEEE Transactions on Industrial Electronics.

[24]  Xingjian Jing.  An H∞ control approach to robust learning of feedforward neural networks , 2011, Neural Networks.

[25]  Hao Yu,et al.  Advantages of Radial Basis Function Networks for Dynamic System Design , 2011, IEEE Transactions on Industrial Electronics.

[26]  Adam Slowik,et al.  Application of an Adaptive Differential Evolution Algorithm With Multiple Trial Vectors to Artificial Neural Network Training , 2011, IEEE Transactions on Industrial Electronics.

[27]  Mojtaba Ahmadieh Khanesar,et al.  Extended Kalman Filter Based Learning Algorithm for Type-2 Fuzzy Logic Systems and Its Experimental Evaluation , 2012, IEEE Transactions on Industrial Electronics.

[28]  Kok Kiong Tan,et al.  Intelligent Friction Modeling and Compensation Using Neural Network Approximations , 2012, IEEE Transactions on Industrial Electronics.

[29]  Okyay Kaynak,et al.  Sliding Mode Control Approach for Online Learning as Applied to Type-2 Fuzzy Neural Networks and Its Experimental Evaluation , 2012, IEEE Transactions on Industrial Electronics.