An artificial modular neural network and its basic dynamical characteristics

Abstract. This work contains a proposition of an artificial modular neural network (MNN) in which every module network exchanges input/output information with others simultaneously. It further studies the basic dynamical characteristics of this network through both computer simulations and analytical considerations. A notable feature of this model is that it has generic representation with regard to the number of composed modules, network topologies, and classes of introduced interactions. The information processing of the MNN is described as the minimization of a total-energy function that consists of partial-energy functions for modules and their interactions, and the activity and weight dynamics are derived from the total-energy function under the Lyapunov stability condition. This concept was realized by Cross-Coupled Hopfield Nets (CCHN) that one of the authors proposed. In this paper, in order to investigate the basic dynamical properties of CCHN, we offer a representative model called Cross-Coupled Hopfield Nets with Local And Global Interactions (CCHN-LAGI) to which two distinct classes of interactions – local and global interactions – are introduced. Through a conventional test for associative memories, it is confirmed that our energy-function-based approach gives us proper dynamics of CCHN-LAGI even if the networks have different modularity. We also discuss the contribution of a single interaction and the joint contribution of the two distinct interactions through the eigenvalue analysis of connection matrices.

[1]  Robert A. Jacobs,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1993, Neural Computation.

[2]  Kazuyoshi Tsutsumi,et al.  The estimation of cross-coupled Hopfield nets as an interactive modular neural network , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[3]  M S Gazzaniga,et al.  Organization of the human brain. , 1989, Science.

[4]  R. H. Phaf,et al.  CALM: Categorizing and learning module , 1992, Neural Networks.

[5]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[6]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[7]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[8]  Toshiki Kindo.  A geometrical analysis of the dynamics of associative memory , 1997 .

[9]  Kunihiko Fukushima,et al.  Recognition and segmentation of connected characters with selective attention , 1993, Neural Networks.

[10]  S. Shipp,et al.  The functional logic of cortical connections , 1988, Nature.

[11]  Kazuyoshi Tsutsumi,et al.  Higher degree error backpropagation in cross-coupled Hopfield nets , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[12]  Bart L. M. Happel,et al.  Design and evolution of modular neural network architectures , 1994, Neural Networks.

[13]  Alexander H. Waibel,et al.  Modular Construction of Time-Delay Neural Networks for Speech Recognition , 1989, Neural Computation.

[14]  Tom Tollenaere,et al.  A modular artificial neural network for texture processing , 1993, Neural Networks.

[15]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[16]  Mahesan Niranjan,et al.  A theoretical investigation into the performance of the Hopfield model , 1990, IEEE Trans. Neural Networks.

[17]  Kazuyoshi Tsutsumi,et al.  Neural computation for controlling the configuration of 2-dimensional truss structure , 1988, IEEE 1988 International Conference on Neural Networks.

[18]  Teuvo Kohonen,et al.  Self-organization and associative memory: 3rd edition , 1989 .

[19]  K. Tsutsumi.  A multilayered neural network composed of backpropagation and Hopfield nets and internal space representation , 1989, International 1989 Joint Conference on Neural Networks.

[20]  K. Tsutsumi,et al.  Cross-coupled Hopfield nets via generalized-delta-rule-based internetworks , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[21]  Santosh S. Venkatesh,et al.  The capacity of the Hopfield associative memory , 1987, IEEE Trans. Inf. Theory.

[22]  Mitsuo Kawato,et al.  Recognition of manipulated objects by motor learning with modular architecture networks , 1991, Neural Networks.

[23]  Stephen Grossberg,et al.  The ART of adaptive pattern recognition by a self-organizing neural network , 1988, Computer.

[24]  Mohamad H. Hassoun.  Dynamic heteroassociative neural memories , 1989, Neural Networks.

[25]  Michael I. Jordan,et al.  Learning piecewise control strategies in a modular neural network architecture , 1993, IEEE Trans. Syst. Man Cybern..

[26]  Bart Kosko,et al.  Unsupervised learning in noise , 1990, International 1989 Joint Conference on Neural Networks.

[27]  Kazuyoshi Tsutsumi,et al.  A multimodule neural network model and the estimate of its nature as associative memory , 1995, Systems and Computers in Japan.

[28]  Minami Ito,et al.  Columns for visual features of objects in monkey inferotemporal cortex , 1992, Nature.

[29]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .