A Self Organizing Recurrent Neural Network

A recurrent neural network with a self-organizing structure based on the dynamic analysis of a task is presented in this paper. The stability of the recurrent neural network is guaranteed by design. A dynamic analysis method to sequence the subsystems of the recurrent neural network according to the fitness between the subsystems and the target system is developed. The network is trained with the network's structure self-organized by dynamically activating subsystems of the network according to tasks. The experiments showed the proposed network is capable of activating appropriate subsystems to approximate different nonlinear dynamic systems regardless of the inputs. When the network was applied to the problem of simultaneously soft measuring the chemical oxygen demand (COD) and NH3-N in wastewater treatment process, it showed its ability of avoiding the coupling influence of the two parameters and thus achieved a more desirable outcome.

[1]  Narasimhan Sundararajan,et al.  A Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks , 2006, IEEE Transactions on Neural Networks.

[2]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[3]  Min Wu,et al.  An improved global asymptotic stability criterion for delayed cellular neural networks , 2006, IEEE Transactions on Neural Networks.

[4]  Girijesh Prasad,et al.  Faster Self-Organizing Fuzzy Neural Network Training and Improved Autonomy withTime-Delayed Synapses for Locally Recurrent Learning , 2010 .

[5]  Magdi S. Mahmoud,et al.  Stability of Discrete Recurrent Neural Networks with Interval Delays: Global Results , 2012, Int. J. Syst. Dyn. Appl..

[6]  Jinde Cao,et al.  Global exponential stability of discrete-time recurrent neural network for solving quadratic programming problems subject to linear constraints , 2011, Neurocomputing.

[7]  Jochen J. Steil,et al.  Online reservoir adaptation by intrinsic plasticity for backpropagation-decorrelation and echo state learning , 2007, Neural Networks.

[8]  Gordon Pipa,et al.  SORN: A Self-Organizing Recurrent Neural Network , 2009, Front. Comput. Neurosci..

[9]  L. Glass,et al.  Oscillation and chaos in physiological control systems. , 1977, Science.

[10]  Andrew Chi-Sing Leung,et al.  Combined learning and pruning for recurrent radial basis function networks based on recursive least square algorithms , 2005, Neural Computing & Applications.

[11]  Jang-Hyun Park,et al.  Direct adaptive controller for nonaffine nonlinear systems using self-structuring neural networks , 2005, IEEE Transactions on Neural Networks.

[12]  Junfei Qiao,et al.  A Stable Online Self-Constructing Recurrent Neural Network , 2011, ISNN.

[13]  Junfei Qiao,et al.  Research on an online self-organizing radial basis function neural network , 2010, Neural Computing and Applications.

[14]  Kumpati S. Narendra,et al.  Identification and control of dynamical systems using neural networks , 1990, IEEE Trans. Neural Networks.

[15]  T. Martin McGinnity,et al.  Faster Self-Organizing Fuzzy Neural Network Training and a Hyperparameter Analysis for a Brain–Computer Interface , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[16]  Florentin Wörgötter,et al.  Information Theoretic Self-organised Adaptation in Reservoirs for Temporal Memory Tasks , 2012, EANN.

[17]  Shyamala C. Sivakumar,et al.  Online stabilization of block-diagonal recurrent neural networks , 1999, IEEE Trans. Neural Networks.

[18]  Narasimhan Sundararajan,et al.  A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation , 2005, IEEE Transactions on Neural Networks.

[19]  Alberto Del Bimbo,et al.  Block-structured recurrent neural networks , 1995, Neural Networks.

[20]  Paris A. Mastorocostas,et al.  A stable learning algorithm for block-diagonal recurrent neural networks: application to the analysis of lung sounds , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[21]  Lee A. Feldkamp,et al.  Neurocontrol of nonlinear dynamical systems with Kalman filter trained recurrent networks , 1994, IEEE Trans. Neural Networks.

[22]  K. Patan.  Stability Analysis and the Stabilization of a Class of Discrete-Time Dynamic Neural Networks , 2007, IEEE Transactions on Neural Networks.

[23]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[24]  Ah Chung Tsoi,et al.  Discrete time recurrent neural network architectures: A unifying review , 1997, Neurocomputing.

[25]  Jeen-Shing Wang,et al.  A Wiener-type recurrent neural network and its control strategy for nonlinear dynamic applications , 2009 .

[26]  Barak A. Pearlmutter.  Gradient calculations for dynamic recurrent neural networks: a survey , 1995, IEEE Trans. Neural Networks.

[27]  Florentin Wörgötter,et al.  Information dynamics based self-adaptive reservoir for delay temporal memory tasks , 2013, Evol. Syst..

[28]  Minoru Asada,et al.  Initialization and self‐organized optimization of recurrent neural network connectivity , 2009, HFSP journal.

[29]  Wei Xing Zheng,et al.  A new approach to stability analysis of discrete-time recurrent neural networks with time-varying delay , 2009, Neurocomputing.

[30]  Marios M. Polycarpou,et al.  High-order neural network structures for identification of dynamical systems , 1995, IEEE Trans. Neural Networks.