Fuzzy CMAC With Incremental Bayesian Ying–Yang Learning and Dynamic Rule Construction

Inspired by the philosophy of ancient Chinese Taoism, Xu's Bayesian ying-yang (BYY) learning technique performs clustering by harmonizing the training data (yang) with the solution (ying). In our previous work, the BYY learning technique was applied to a fuzzy cerebellar model articulation controller (FCMAC) to find the optimal fuzzy sets; however, this is not suitable for time series data analysis. To address this problem, we propose an incremental BYY learning technique in this paper, with the idea of sliding window and rule structure dynamic algorithms. Three contributions are made as a result of this research. First, an online expectation-maximization algorithm incorporated with the sliding window is proposed for the fuzzification phase. Second, the memory requirement is greatly reduced since the entire data set no longer needs to be obtained during the prediction process. Third, the rule structure dynamic algorithm with dynamically initializing, recruiting, and pruning rules relieves the ¿curse of dimensionality¿ problem that is inherent in the FCMAC. Because of these features, the experimental results of the benchmark data sets of currency exchange rates and Mackey-Glass show that the proposed model is more suitable for real-time streaming data analysis.

[1]  James S. Albus,et al.  New Approach to Manipulator Control: The Cerebellar Model Articulation Controller (CMAC)1 , 1975 .

[2]  L. Glass,et al.  Oscillation and chaos in physiological control systems. , 1977, Science.

[3]  John C. Platt A Resource-Allocating Network for Function Interpolation , 1991, Neural Computation.

[4]  Bernd Fritzke,et al.  A Growing Neural Gas Network Learns Topologies , 1994, NIPS.

[5]  Qing Zhu,et al.  Fuzzy and Evidence Reasoning , 1995 .

[6]  Chin-Teng Lin,et al.  An online self-constructing neural fuzzy inference network and its applications , 1998, IEEE Trans. Fuzzy Syst..

[7]  Kevin P. Murphy,et al.  Bayesian Map Learning in Dynamic Environments , 1999, NIPS.

[8]  Volker Tresp,et al.  Robust neural network online learning in time-variant regression models , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[9]  Nikola K. Kasabov,et al.  ESOM: an algorithm to evolve self-organizing maps from online data streams , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[10]  Nikola K. Kasabov,et al.  Evolving fuzzy neural networks for supervised/unsupervised online knowledge-based learning , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[11]  Amit Mitra,et al.  Forecasting daily foreign exchange rates using genetically optimized neural networks , 2002 .

[12]  Piotr Indyk,et al.  Maintaining stream statistics over sliding windows: (extended abstract) , 2002, SODA '02.

[13]  Rajeev Motwani,et al.  Sampling from a moving window over streaming data , 2002, SODA '02.

[14]  Piotr Indyk,et al.  Maintaining Stream Statistics over Sliding Windows , 2002, SIAM J. Comput..

[15]  Nikola K. Kasabov,et al.  DENFIS: dynamic evolving neural-fuzzy inference system and its application for time-series prediction , 2002, IEEE Trans. Fuzzy Syst..

[16]  Lei Xu,et al.  2002 Special Issue BYY harmony learning, structural RPCL, and topological self-organizing on mixture models q , 2002 .

[17]  Lei Xu,et al.  BYY harmony learning, structural RPCL, and topological self-organizing on mixture models , 2002, Neural Networks.

[18]  Tim Brailsford,et al.  Selecting the forgetting factor in subset autoregressive modelling , 2002 .

[19]  V. Kazakevičius,et al.  Stability of random coefficient ARCH models and aggregation schemes , 2004 .

[20]  Gábor Horváth Kernel CMAC with improved capability , 2004 .

[21]  Lei Xu,et al.  Advances on BYY harmony learning: information theoretic perspective, generalized projection geometry, and independent factor autodetermination , 2004, IEEE Transactions on Neural Networks.

[22]  Daming Shi,et al.  Sensitivity analysis applied to the construction of radial basis function networks , 2005, Neural Networks.

[23]  Daming Shi,et al.  FCMAC-BYY: Fuzzy CMAC Using Bayesian Ying–Yang Learning , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[24]  Dipti Srinivasan,et al.  Neural Networks for Continuous Online Learning and Control , 2006, IEEE Transactions on Neural Networks.

[25]  Daming Shi,et al.  Product Demand Forecasting with a Novel Fuzzy CMAC , 2007, Neural Processing Letters.

[26]  Edwin Lughofer,et al.  FLEXFIS: A Robust Incremental Learning Approach for Evolving Takagi–Sugeno Fuzzy Models , 2008, IEEE Transactions on Fuzzy Systems.

[27]  Daming Shi,et al.  An online Bayesian Ying-Yang learning applied to fuzzy CMAC , 2008, Neurocomputing.