An online self-adaptive modular neural network for time-varying systems

We propose an online self-adaptive modular neural network (OSAMNN) for time-varying systems. Starting with zero subnetworks, OSAMNN uses a single-pass subtractive cluster algorithm to update the centers of radial-basis function (RBF) neurons for learning. Then the input space can be partitioned. The OSAMNN structure is capable of growing or merging subnetworks to maintain suitable model complexity, and the centers of RBF neurons can also be dynamically adjusted according to changes in the data environment. A fuzzy strategy is applied to select suitable subnetworks to learn the current sample. This method yields improved learning efficiency and accuracy. OSAMNN can adapt its architecture to realize online modeling of time-varying nonlinear input-output maps. Results for experiments on benchmark and real-world time-varying systems support the proposed techniques.

[1]  Karina Gibert,et al.  Knowledge discovery with clustering based on rules by states: A water treatment application , 2010, Environ. Model. Softw..

[2]  Mohamed S. Kamel,et al.  Modular neural networks: a survey. , 1999, International journal of neural systems.

[3]  Wei Tang,et al.  Ensembling neural networks: Many could be better than all , 2002, Artif. Intell..

[4]  Licheng Jiao,et al.  Adaptive Tracking for Periodically Time-Varying and Nonlinearly Parameterized Systems Using Multilayer Neural Networks , 2010, IEEE Transactions on Neural Networks.

[5]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[6]  Nikola Gradojevic,et al.  Option Pricing With Modular Neural Networks , 2009, IEEE Transactions on Neural Networks.

[7]  Eric Ronco,et al.  Modular Neural Networks: a state of the art , 1995 .

[8]  Farooq Azam,et al.  Biologically Inspired Modular Neural Networks , 2000 .

[9]  Robi Polikar,et al.  Incremental Learning of Concept Drift in Nonstationary Environments , 2011, IEEE Transactions on Neural Networks.

[10]  Juan López Coronado,et al.  A modular neural network architecture for step-wise learning of grasping tasks , 2007, Neural Networks.

[11]  José Manuel Gutiérrez,et al.  Evolving modular networks with genetic algorithms: application to nonlinear time series , 2004, Expert Syst. J. Knowl. Eng..

[12]  D.P. Filev,et al.  An approach to online identification of Takagi-Sugeno fuzzy models , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[13]  Rui Xu,et al.  Survey of clustering algorithms , 2005, IEEE Transactions on Neural Networks.

[14]  John Yen,et al.  Extracting fuzzy rules for system modeling using a hybrid of genetic algorithms and Kalman filter , 1999, Fuzzy Sets Syst..

[15]  Paramasivan Saratchandran,et al.  Sequential Adaptive Fuzzy Inference System (SAFIS) for nonlinear system identification and prediction , 2006, Fuzzy Sets Syst..

[16]  O Yamanaka,et al.  Total cost minimization control scheme for biological wastewater treatment process and its evaluation based on the COST benchmark process. , 2006, Water science and technology : a journal of the International Association on Water Pollution Research.

[17]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[18]  L M HappelBart,et al.  1994 Special Issue , 1994 .

[19]  Chu Kiong Loo,et al.  Novel direct and self-regulating approaches to determine optimum growing multi-experts network structure , 2004, IEEE Transactions on Neural Networks.

[20]  Jonas Balderud,et al.  Data-driven adaptive model-based predictive control with application in wastewater systems , 2011 .

[21]  Jianbo Liu,et al.  Topology Preservation and Cooperative Learning in Identification of Multiple Model Systems , 2008, IEEE Transactions on Neural Networks.

[22]  Bart L. M. Happel,et al.  Design and evolution of modular neural network architectures , 1994, Neural Networks.

[23]  Hiok Chai Quek,et al.  A BCM Theory of Meta-Plasticity for Online Self-Reorganizing Fuzzy-Associative Learning , 2010, IEEE Transactions on Neural Networks.

[24]  Xuemei Ren,et al.  Neural Networks-Based Adaptive Control for Nonlinear Time-Varying Delays Systems With Unknown Control Direction , 2011, IEEE Transactions on Neural Networks.

[25]  Nikola K. Kasabov,et al.  DENFIS: dynamic evolving neural-fuzzy inference system and its application for time-series prediction , 2002, IEEE Trans. Fuzzy Syst..

[26]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[27]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[28]  Joydeep Ghosh,et al.  Structurally adaptive modular networks for nonstationary environments , 1999, IEEE Trans. Neural Networks.

[29]  Plamen P. Angelov,et al.  Adaptive Inferential Sensors Based on Evolving Fuzzy Models , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[30]  Ke-Lin Du,et al.  Clustering: A neural network approach , 2010, Neural Networks.

[31]  Gordon Lightbody,et al.  Local Model Network Identification With Gaussian Processes , 2007, IEEE Transactions on Neural Networks.

[32]  Huaguang Zhang,et al.  Motif discoveries in unaligned molecular sequences using self-organizing neural networks , 2006, IEEE Trans. Neural Networks.

[33]  S. C. Tong,et al.  Adaptive Neural Network Decentralized Backstepping Output-Feedback Control for Nonlinear Large-Scale Systems With Time Delays , 2011, IEEE Transactions on Neural Networks.

[34]  H. Chris Tseng,et al.  Modular neural networks with applications to pattern profiling problems , 2009, Neurocomputing.

[35]  Chee Kheong Siew,et al.  Incremental extreme learning machine with fully complex hidden nodes , 2008, Neurocomputing.

[36]  L. Glass,et al.  Oscillation and chaos in physiological control systems. , 1977, Science.

[37]  Noel E. Sharkey,et al.  An Analysis of Catastrophic Interference , 1995, Connect. Sci..

[38]  Ke Chen,et al.  A self-generating modular neural network architecture for supervised learning , 1997, Neurocomputing.