Optimization of Neural Networks Using Variable Structure Systems

This paper proposes a new mixed training algorithm consisting of error backpropagation (EBP) and variable structure systems (VSSs) to optimize parameter updating of neural networks. For the optimization of the number of neurons in the hidden layer, a new term based on the output of the hidden layer is added to the cost function as a penalty term to make optimal use of hidden units related to weights corresponding to each unit in the hidden layer. VSS is used to control the dynamic model of the training process, whereas EBP attempts to minimize the cost function. In addition to the analysis of the imposed dynamics of the EBP technique, the global stability of the mixed training methodology and constraints on the design parameters are considered. The advantages of the proposed technique are guaranteed convergence, improved robustness, and lower sensitivity to initial weights of the neural network.

[1]  Bertrand Tondu,et al.  Practical design of real time VSS applied for flexibal robot , 2010, 2010 11th International Workshop on Variable Structure Systems (VSS).

[2]  James T. Kwok,et al.  Constructive algorithms for structure learning in feedforward neural networks for regression problems , 1997, IEEE Trans. Neural Networks.

[3]  Barbara Hammer,et al.  Neural Smithing – Supervised Learning in Feedforward Artificial Neural Networks , 2001, Pattern Analysis & Applications.

[4]  James E. Bobrow,et al.  Adaptive, High Bandwidth Control of a Hydraulic Actuator , 1996 .

[5]  G. W. Ng.  application of Neural Networks to Adaptive Control of Nonlinear Systems , 1997 .

[6]  Lorien Y. Pratt,et al.  Comparing Biases for Minimal Network Construction with Back-Propagation , 1988, NIPS.

[7]  H.-K. Chen.  CHAOS AND CHAOS SYNCHRONIZATION OF A SYMMETRIC GYRO WITH LINEAR-PLUS-CUBIC DAMPING , 2002 .

[8]  Gene H. Golub,et al.  Generalized cross-validation as a method for choosing a good ridge parameter , 1979, Milestones in Matrix Computation.

[9]  M. Teshnehlab,et al.  EHSS Velocity Control by Fuzzy Neural Networks , 2007, NAFIPS 2007 - 2007 Annual Meeting of the North American Fuzzy Information Processing Society.

[10]  Hyungsuck Cho,et al.  Adaptive model following control of electrohydraulic velocity control systems subjected to unknown disturbances , 1988 .

[11]  Septimiu E. Salcudean,et al.  On the nonlinear control of hydraulic servo-systems , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[12]  D. Mayne.  Nonlinear and Adaptive Control Design [Book Review] , 1996, IEEE Transactions on Automatic Control.

[13]  M. Teshnehlab,et al.  Velocity control of an electro hydraulic servomotor by neural networks , 2005, Proceedings. 2005 International Conference Physics and Control, 2005..

[14]  Weibing Gao,et al.  Variable structure control of nonlinear systems: a new approach , 1993, IEEE Trans. Ind. Electron..

[15]  M. Teshnehlab,et al.  Decoupled sliding-mode with fuzzy neural network controller for EHSS velocity control , 2007, 2007 International Conference on Intelligent and Advanced Systems.

[16]  Russell Reed,et al.  Pruning algorithms-a survey , 1993, IEEE Trans. Neural Networks.

[17]  Jun-Juh Yan,et al.  Controlling chaos of a chaotic nonlinear gyro using variable structure control , 2007 .

[18]  Juhng-Perng Su,et al.  A Combined Hard and Soft Variable-Structure Control Scheme for a Class of Nonlinear Systems , 2009, IEEE Transactions on Industrial Electronics.

[19]  Okyay Kaynak,et al.  Applications of VSC in motion control systems , 1994 .

[20]  E. Ryan.  A variable structure approach to feedback regulation of uncertain dynamical systems , 1983 .

[21]  E. Fiesler,et al.  Comparative Bibliography of Ontogenic Neural Networks , 1994 .

[22]  S. K. Rogers,et al.  A taxonomy of neural network optimality , 1992, Proceedings of the IEEE 1992 National Aerospace and Electronics Conference@m_NAECON 1992.

[23]  Vadim I. Utkin,et al.  Sliding Modes and their Application in Variable Structure Systems , 1978 .

[24]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[25]  John Y. Hung,et al.  Variable structure control: a survey , 1993, IEEE Trans. Ind. Electron..

[26]  B. Bandyopadhyay,et al.  Design of power system stabilizer using power rate reaching law based sliding mode control technique , 2005, 2005 International Power Engineering Conference.

[27]  Septimiu E. Salcudean,et al.  Modeling, simulation, and control of a hydraulic Stewart platform , 1997, Proceedings of International Conference on Robotics and Automation.

[28]  M. Spong,et al.  Robust Control Design Techniques for a Class of Nonlinear Systems , 1986, 1986 American Control Conference.

[29]  Okyay Kaynak,et al.  The fusion of computationally intelligent methodologies and sliding-mode control-a survey , 2001, IEEE Trans. Ind. Electron..

[30]  Wang Tong,et al.  Variable structure approach power compensation system design of an automatic carrier landing system , 2009, 2009 Chinese Control and Decision Conference.

[31]  K. D. Young,et al.  Design of variable structure model-following control systems , 1978 .

[32]  Wu-Chung Su,et al.  Reconstructing Sliding Surfaces for SISO Variable Structure Output Feedback Control Systems Using Inverse Method , 2010, IEEE Transactions on Automatic Control.

[33]  Madan M. Gupta,et al.  Static and Dynamic Neural Networks: From Fundamentals to Advanced Theory , 2003 .

[34]  David E. Rumelhart,et al.  Generalization by Weight-Elimination with Application to Forecasting , 1990, NIPS.

[35]  M. Jovanovic,et al.  Nonlinear control of an electrohydraulic velocity servosystem , 2002, Proceedings of the 2002 American Control Conference (IEEE Cat. No.CH37301).