Nonlinear System Modelling Via Optimal Design Of Neural Trees

This paper introduces a flexible neural tree model. The model is computed as a flexible multi-layer feed-forward neural network. A hybrid learning/evolutionary approach to automatically optimize the neural tree model is also proposed. The approach includes a modified probabilistic incremental program evolution algorithm (MPIPE) to evolve and determine a optimal structure of the neural tree and a parameter learning algorithm to optimize the free parameters embedded in the neural tree. The performance and effectiveness of the proposed method are evaluated using function approximation, time series prediction and system identification problems and compared with the related methods.

[1]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1972 .

[2]  R. Tong.  The evaluation of fuzzy models derived from experimental data , 1980 .

[3]  W. Pedrycz.  An identification algorithm in fuzzy relational systems , 1984 .

[4]  Yong-Zai Lu,et al.  Fuzzy Model Identification and Self-Learning for Dynamic Systems , 1987, IEEE Transactions on Systems, Man, and Cybernetics.

[5]  Jean-Pierre Nadal,et al.  Study of a Growth Algorithm for a Feedforward Network , 1989, Int. J. Neural Syst..

[6]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[7]  Peter M. Todd,et al.  Designing Neural Networks using Genetic Algorithms , 1989, ICGA.

[8]  L. Darrell Whitley,et al.  Genetic algorithms and neural networks: optimizing connections and connectivity , 1990, Parallel Comput..

[9]  Hiroaki Kitano,et al.  Designing Neural Networks Using Genetic Algorithms with Graph Generation System , 1990, Complex Syst..

[10]  Michio Sugeno,et al.  A fuzzy-logic-based approach to qualitative modeling , 1993, IEEE Trans. Fuzzy Syst..

[11]  H. Surmann,et al.  Self-Organizing and Genetic Algorithms for an Automatic Design of Fuzzy Control and Decision Systems , 1993 .

[12]  Frédéric Gruau,et al.  Genetic Synthesis of Modular Neural Networks , 1993, ICGA.

[13]  Donald E. Waagen,et al.  Evolving recurrent perceptrons for time-series modeling , 1994, IEEE Trans. Neural Networks.

[14]  C. Hwang,et al.  A Combined Approach to Fuzzy Model Identification , 1994, IEEE Trans. Syst. Man Cybern. Syst..

[15]  Peter J. Angeline,et al.  An evolutionary algorithm that constructs recurrent neural networks , 1994, IEEE Trans. Neural Networks.

[16]  Vasant Honavar,et al.  Evolutionary Design of Neural Architectures , 1995 .

[17]  David B. Fogel,et al.  Evolving Neural Control Systems , 1995, IEEE Expert.

[18]  Junhong Nie,et al.  Constructing fuzzy model by self-organizing counterpropagation network , 1995, IEEE Trans. Syst. Man Cybern..

[19]  Yinghua Lin,et al.  A new approach to fuzzy-neural system modeling , 1995, IEEE Trans. Fuzzy Syst..

[20]  Rudy Setiono,et al.  Use of a quasi-Newton method in a feedforward neural network construction algorithm , 1995, IEEE Trans. Neural Networks.

[21]  Marzuki Khalid,et al.  Neuro-control and its applications , 1996 .

[22]  Shyh Hwang,et al.  An identification algorithm in fuzzy relational systems , 1996, Soft Computing in Intelligent Systems and Information Processing. Proceedings of the 1996 Asian Fuzzy Systems Symposium.

[23]  Michael J. Watts,et al.  FuNN/2 - A Fuzzy Neural Network Architecture for Adaptive Learning and Knowledge Acquisition , 1997, Inf. Sci..

[24]  Yinghua Lin,et al.  Using fuzzy partitions to create fuzzy systems from input-output data and set the initial weights in a fuzzy neural network , 1997, IEEE Trans. Fuzzy Syst..

[25]  Rafal Salustowicz,et al.  Probabilistic Incremental Program Evolution , 1997, Evolutionary Computation.

[26]  Xin Yao,et al.  A new evolutionary system for evolving artificial neural networks , 1997, IEEE Trans. Neural Networks.

[27]  Byoung-Tak Zhang,et al.  Evolutionary Induction of Sparse Neural Trees , 1997, Evolutionary Computation.

[28]  Antonio F. Gómez-Skarmeta,et al.  A fuzzy clustering-based rapid prototyping for fuzzy rule-based modeling , 1997, IEEE Trans. Fuzzy Syst..

[29]  Chuen-Tsai Sun,et al.  Neuro-fuzzy And Soft Computing: A Computational Approach To Learning And Machine Intelligence [Books in Brief] , 1997, IEEE Transactions on Neural Networks.

[30]  Kotaro Hirasawa,et al.  RasID - Random Search for Neural Network Training , 1998, J. Adv. Comput. Intell. Intell. Informatics.

[31]  Yuehui Chen,et al.  Evolutionary control of discrete-time nonlinear system using PIPE algorithm , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[32]  Nikola K. Kasabov,et al.  HyFIS: adaptive neuro-fuzzy inference systems and their application to nonlinear dynamical systems , 1999, Neural Networks.

[33]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[34]  Xin Yao,et al.  Evolutionary programming made faster , 1999, IEEE Trans. Evol. Comput..

[35]  O. Nelles.  Nonlinear System Identification: From Classical Approaches to Neural Networks and Fuzzy Models , 2000 .

[36]  Marco Russo,et al.  Genetic fuzzy learning , 2000, IEEE Trans. Evol. Comput..

[37]  Yuehui Chen,et al.  System Identification and Control using Probabilistic Incremental Program Evolution Algorithm , 2000, J. Robotics Mechatronics.

[38]  Hartmut Surmann,et al.  Learning feed-forward and recurrent fuzzy systems: A genetic approach , 2001, J. Syst. Archit..

[39]  Yuehui Chen,et al.  Evolving Neurofuzzy System by Hybrid Soft Computing Approaches for System Identification , 2001, J. Adv. Comput. Intell. Intell. Informatics.

[40]  Yuehui Chen,et al.  Evolving Basis Function Networks for System Identification , 2001, J. Adv. Comput. Intell. Intell. Informatics.

[41]  Byoung-Tak Zhang.  A Bayesian evolutionary approach to the design and learning of heterogeneous neural trees , 2002, Integr. Comput. Aided Eng..

[42]  Cândida Ferreira,et al.  Genetic Representation and Genetic neutrality in gene Expression Programming , 2002, Adv. Complex Syst..

[43]  Dragan Kukolj,et al.  Design of adaptive Takagi-Sugeno-Kang fuzzy models , 2002, Appl. Soft Comput..

[44]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[45]  D. B. Fogel,et al.  Evolving neural networks , 1990, Biological Cybernetics.