Robustly Fitting and Forecasting Dynamical Data With Electromagnetically Coupled Artificial Neural Network: A Data Compression Method

In this paper, a dynamical recurrent artificial neural network (ANN) is proposed and studied. Inspired from a recent research in neuroscience, we introduced nonsynaptic coupling to form a dynamical component of the network. We mathematically proved that, with adequate neurons provided, this dynamical ANN model is capable of approximating any continuous dynamic system with an arbitrarily small error in a limited time interval. Its extreme concise Jacobian matrix makes the local stability easy to control. We designed this ANN for fitting and forecasting dynamic data and obtained satisfied results in simulation. The fitting performance is also compared with those of both the classic dynamic ANN and the state-of-the-art models. Sufficient trials and the statistical results indicated that our model is superior to those have been compared. Moreover, we proposed a robust approximation problem, which asking the ANN to approximate a cluster of input–output data pairs in large ranges and to forecast the output of the system under previously unseen input. Our model and learning scheme proposed in this paper have successfully solved this problem, and through this, the approximation becomes much more robust and adaptive to noise, perturbation, and low-order harmonic wave. This approach is actually an efficient method for compressing massive external data of a dynamic system into the weight of the ANN.

[1]  Indranil Saha,et al.  journal homepage: www.elsevier.com/locate/neucom , 2022 .

[2]  T. Poggio,et al.  General conditions for predictivity in learning theory , 2004, Nature.

[3]  Gaute T. Einevoll,et al.  Frequency Dependence of Signal Power and Spatial Reach of the Local Field Potential , 2013, PLoS Comput. Biol..

[4]  Daniel T. Larose,et al.  Discovering Knowledge in Data: An Introduction to Data Mining , 2005 .

[5]  G. Lewicki,et al.  Approximation by Superpositions of a Sigmoidal Function , 2003 .

[6]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[7]  Qi-Jun Zhang,et al.  State-space dynamic neural network technique for high-speed IC applications: modeling and stability analysis , 2006, IEEE Transactions on Microwave Theory and Techniques.

[8]  Marios M. Polycarpou,et al.  High-order neural network structures for identification of dynamical systems , 1995, IEEE Trans. Neural Networks.

[9]  Ajay M. Patrikar.  Approximating Gaussian Mixture Model or Radial Basis Function Network With Multilayer Perceptron , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[10]  Christof Koch,et al.  The Spiking Component of Oscillatory Extracellular Potentials in the Rat Hippocampus , 2012, The Journal of Neuroscience.

[11]  Jianjun Xu,et al.  Neural based dynamic modeling of nonlinear microwave circuits , 2002, IMS 2002.

[12]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1990 .

[13]  M. Fox,et al.  Spontaneous fluctuations in brain activity observed with functional magnetic resonance imaging , 2007, Nature Reviews Neuroscience.

[14]  Kenneth D. Miller,et al.  Mathematical Equivalence of Two Common Forms of Firing Rate Models of Neural Networks , 2012, Neural Computation.

[15]  Richard D. Braatz,et al.  Universal approximation with error bounds for dynamic artificial neural network models: A tutorial and some new results , 2011, 2011 IEEE International Symposium on Computer-Aided Control System Design (CACSD).

[17]  Leon O. Chua,et al.  Cellular neural networks: applications , 1988 .

[18]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[19]  Christof Koch,et al.  Ephaptic coupling of cortical neurons , 2011, Nature Neuroscience.

[20]  Anil K. Jain,et al.  Artificial Neural Networks: A Tutorial , 1996, Computer.

[21]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[22]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[23]  Siegel Markus,et al.  Oscillatory synchronization in large-scale cortical networks predicts perception , 2011 .

[24]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[25]  J. Hale,et al.  Ordinary Differential Equations , 2019, Fundamentals of Numerical Mathematics for Physicists and Engineers.

[26]  Yuhui Shi,et al.  Particle swarm optimization: developments, applications and resources , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[27]  Ching-Hung Lee,et al.  Identification and control of dynamic systems using recurrent fuzzy neural networks , 2000, IEEE Trans. Fuzzy Syst..