A recurrent fuzzy-neural model for dynamic system identification

This paper presents a fuzzy modeling approach for identification of dynamic systems. In particular, a new fuzzy model, the Dynamic Fuzzy Neural Network (DFNN), consisting of recurrent TSK rules, is developed. The premise and defuzzification parts are static while the consequent parts of the fuzzy rules are recurrent neural networks with internal feedback and time delay synapses. The network is trained by means of a novel learning algorithm, named Dynamic-Fuzzy Neural Constrained Optimization Method (D-FUNCOM), based on the concept of constrained optimization. The proposed algorithm is general since it can be applied to locally as well as fully recurrent networks, regardless of their structures. An adaptation mechanism of the maximum parameter change is presented as well. The proposed dynamic model, equipped with the learning algorithm, is applied to several temporal problems, including modeling of a NARMA process and the noise cancellation problem. Performance comparisons are conducted with a series of static and dynamic systems and some existing recurrent fuzzy models. Simulation results show that DFNN compares favorably with its competing rivals and thus it can be considered for efficient system identification.

[1]  Eric A. Wan,et al.  Temporal backpropagation for FIR neural networks , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[2]  Giovanni L. Sicuranza,et al.  Adaptive echo cancellation with nonlinear digital filters , 1984, ICASSP.

[3]  Ah Chung Tsoi,et al.  Locally recurrent globally feedforward networks: a critical review of architectures , 1994, IEEE Trans. Neural Networks.

[4]  Ching-Hung Lee,et al.  Identification and control of dynamic systems using recurrent fuzzy neural networks , 2000, IEEE Trans. Fuzzy Syst..

[5]  Hideyuki Takagi,et al.  Neural networks designed on approximate reasoning architecture and their applications , 1992, IEEE Trans. Neural Networks.

[6]  Stephen L. Chiu,et al.  Fuzzy Model Identification Based on Cluster Estimation , 1994, J. Intell. Fuzzy Syst..

[7]  George C. Mouzouris,et al.  Dynamic non-Singleton fuzzy logic systems for nonlinear modeling , 1997, IEEE Trans. Fuzzy Syst..

[8]  S. W. Piche,et al.  Steepest descent algorithms for neural network controllers and filters , 1994, IEEE Trans. Neural Networks.

[9]  Kumpati S. Narendra,et al.  Identification and control of dynamical systems using neural networks , 1990, IEEE Trans. Neural Networks.

[10]  Richard S. Sutton,et al.  Neural networks for control , 1990 .

[11]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[12]  S. A. Billings,et al.  Performance monitoring in nonlinear adaptive noise cancellation , 1990 .

[13]  Hugues Bersini,et al.  Recurrent fuzzy systems , 1994, Proceedings of 1994 IEEE 3rd International Fuzzy Systems Conference.

[14]  Stephen A. Billings,et al.  Recurrent radial basis function networks for adaptive noise cancellation , 1995, Neural Networks.

[15]  Michael I. Jordan.  Supervised learning and systems with excess degrees of freedom , 1988 .

[16]  Geoffrey E. Hinton,et al.  Phoneme recognition using time-delay neural networks , 1989, IEEE Trans. Acoust. Speech Signal Process..

[17]  Steve A. Billings,et al.  Performance monitoring in non-linear adaptive noise cancellation , 1992 .

[18]  George Vachtsevanos,et al.  Recursive learning algorithms for training fuzzy recurrent models , 1996 .

[19]  P. Feintuch.  An adaptive recursive LMS filter , 1976, Proceedings of the IEEE.

[20]  M. Gori,et al.  BPS: a learning algorithm for capturing the dynamic nature of speech , 1989, International 1989 Joint Conference on Neural Networks.

[21]  P. S. Sastry,et al.  Memory neuron networks for identification and control of dynamical systems , 1994, IEEE Trans. Neural Networks.

[22]  A. Curtis,et al.  Adaptive noise cancellation for a class of non-linear IIR filters , 1990 .

[23]  Jyh-Shing Roger Jang,et al.  Self-learning fuzzy controllers based on temporal backpropagation , 1992, IEEE Trans. Neural Networks.

[24]  Giovanni Soda,et al.  Local Feedback Multilayered Networks , 1992, Neural Computation.

[25]  Chin-Teng Lin,et al.  A recurrent self-organizing neural fuzzy inference network , 1999, IEEE Trans. Neural Networks.

[26]  A. E. Bryson,et al.  A Steepest-Ascent Method for Solving Optimum Programming Problems , 1962 .

[27]  Jyh-Shing Roger Jang,et al.  ANFIS: adaptive-network-based fuzzy inference system , 1993, IEEE Trans. Syst. Man Cybern..

[28]  I. J. Leontaritis,et al.  Model selection and validation methods for non-linear systems , 1987 .

[29]  Michael J. Coker,et al.  A nonlinear adaptive noise canceller , 1980, ICASSP.

[30]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[31]  S Z Qin,et al.  Comparison of four neural net learning methods for dynamic system identification , 1992, IEEE Trans. Neural Networks.

[32]  Pierre Roussel-Ragot,et al.  Neural Networks and Nonlinear Adaptive Filtering: Unifying Concepts and New Algorithms , 1993, Neural Computation.

[33]  B. Widrow,et al.  Adaptive noise cancelling: Principles and applications , 1975 .

[34]  Yoshiki Uchikawa,et al.  On fuzzy modeling using fuzzy neural networks with the back-propagation algorithm , 1992, IEEE Trans. Neural Networks.

[35]  Jerry M. Mendel,et al.  Fuzzy adaptive filters, with application to nonlinear channel equalization , 1993, IEEE Trans. Fuzzy Syst..

[36]  Chin-Teng Lin,et al.  An adaptive neural fuzzy filter and its applications , 1997, IEEE Trans. Syst. Man Cybern. Part B.

[37]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[38]  Isao Hayashi,et al.  NN-driven fuzzy reasoning , 1991, Int. J. Approx. Reason..

[39]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[40]  Eiji Mizutani,et al.  Coactive neural fuzzy modeling , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[41]  Paris A. Mastorocostas,et al.  FUNCOM: A constrained learning algorithm for fuzzy neural networks , 2000, Fuzzy Sets Syst..

[42]  Ah Chung Tsoi,et al.  FIR and IIR Synapses, a New Neural Network Architecture for Time Series Modeling , 1991, Neural Computation.

[43]  Stephen Grossberg,et al.  Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system , 1991, Neural Networks.

[44]  Jie Zhang,et al.  Recurrent neuro-fuzzy networks for nonlinear process modeling , 1999, IEEE Trans. Neural Networks.

[45]  Dimitris A. Karras,et al.  An efficient constrained training algorithm for feedforward networks , 1995, IEEE Trans. Neural Networks.