On Some Dynamical Properties of Randomly Connected Higher Order Neural Networks

This chapter presents macroscopic properties of higher order neural networks. Randomly connected Neural Networks (RNNs) are known as a convenient model to investigate the macroscopic properties of neural networks. They are investigated by using the statistical method of neuro-dynamics. By applying the approach to higher order neural networks, macroscopic properties of them are made clear. The approach establishes: (a) there are differences between stability of RNNs and Randomly connected Higher Order Neural Networks (RHONNs) in the cases of the digital state − { } 1 1 , -model and the analog state [ , ] −1 1 model; (b) there is no difference between stability of RNNs and RHONNs in the cases of the digital state 0,1 { } -model and the analog state [ , ] 0 1 -model; (c) with neural networks with oscillation, there are large differences between RNNs and RHONNs in the cases of the digital state − { } 1 1 , -model and the analog state [ , ] −1 1 -model, that is, there exists complex dynamics in each model for k = 2 ; (d) behavior of groups composed of RHONNs are represented as a combination of the behavior of each RHONN.

[1]  Shun-ichi Amari,et al.  Characteristics of randomly connected threshold-element networks and network systems , 1971 .

[2]  Paulo J. G. Lisboa,et al.  Translation, rotation, and scale invariant pattern recognition by high-order neural networks and moment classifiers , 1992, IEEE Trans. Neural Networks.

[3]  J. Yorke,et al.  Period Three Implies Chaos , 1975 .

[4]  P.K. Simpson,et al.  Higher-ordered and intraconnected bidirectional associative memories , 1990, IEEE Trans. Syst. Man Cybern..

[5]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[6]  Colin Giles,et al.  Learning, invariance, and generalization in high-order neural networks. , 1987, Applied optics.

[7]  Mingyuan Zhang.  Artificial Higher Order Neural Networks for Modeling and Simulation , 2012 .

[8]  Hiromi Miyajima,et al.  Dynamical properties of higher order random neural networks , 1996, Proceedings Eighth IEEE International Conference on Tools with Artificial Intelligence.

[9]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[10]  Shun-ichi Amari,et al.  A method of statistical neurodynamics , 1974, Kybernetik.

[11]  E M Harth,et al.  Dynamics of neural structures. , 1970, Journal of theoretical biology.

[12]  G. Palm,et al.  On associative memory , 2004, Biological Cybernetics.

[13]  Donald C. Wunsch,et al.  Higher Order Neural Network Architectures for Agent-Based Computational Economics and Finance , 2009 .

[14]  Tong Lee,et al.  On the convergence of neural network for higher order programming , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[15]  J. E. Moreira.  Intermittency in a neural network with variable threshold , 1993 .

[16]  A Hjelmfelt,et al.  Pattern recognition, chaos, and multiplicity in neural networks of excitable systems. , 1994, Proceedings of the National Academy of Sciences of the United States of America.

[17]  Demetri Psaltis,et al.  Higher order associative memories and their optical implementations , 1988, Neural Networks.

[18]  Santosh S. Venkatesh,et al.  The capacity of the Hopfield associative memory , 1987, IEEE Trans. Inf. Theory.

[19]  Ming Zhang.  Artificial Higher Order Neural Networks for Computer Science and Engineering: Trends for Emerging Applications , 2010 .

[20]  Frank L. Lewis,et al.  Back-Stepping Control of Quadrotor: A Dynamically Tuned Higher Order Like Neural Network Approach , 2010 .

[21]  Masahiko Morita,et al.  Associative memory with nonmonotone dynamics , 1993, Neural Networks.

[22]  S. Amari,et al.  Characteristics of Random Nets of Analog Neuron-Like Elements , 1972, IEEE Trans. Syst. Man Cybern..

[23]  Hirofumi Miyajima,et al.  Dynamical properties of neural networks with product connections , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[24]  Shun-ichi Amari,et al.  Statistical neurodynamics of associative memory , 1988, Neural Networks.

[25]  Ming Zhang,et al.  Artificial Multi-Polynomial Higher Order Neural Network Models , 2013 .

[26]  Noritaka Shigei,et al.  On Some Dynamical Properties of Threshold and Homogeneous Networks (Special Section on Nonlinear Theory and Its Applications) , 1994 .