Distributed memory systems for simulating artificial neural networks

In executing tasks involving intelligent information processing, the human brain performs better than the digital computer. The human brain derives its power from a large number [O(1011)] of neurons which are interconnected by a dense interconnection network [O(105) connections per neuron]. Artificial neural network (ANN) paradigms adopt the structure of the brain to try to emulate the intelligent information processing methods of the brain. ANN techniques are being employed to solve problems in areas such as pattern recognition, and robotic processing. Simulation of ANNs involves implementation of large number of neurons and a massive interconnection network. In this paper, we discuss various simulation models of ANNs and their implementation on distributed memory systems. Our investigations reveal that communication-efficient networks of distributed memory systems perform better than other topologies in implementing ANNs.

[1]  Takayuki Ito,et al.  Realization of a Neural Network Model Neocognitron on a Hypercube Parallel Computer , 1990, Int. J. High Speed Comput..

[2]  Jirí Benes,et al.  On neural networks , 1990, Kybernetika.

[3]  Paul J. Werbos,et al.  Backpropagation Through Time: What It Does and How to Do It , 1990, Proc. IEEE.

[4]  Seungryoul Maeng,et al.  Parallel simulation of multilayered neural networks on distributed-memory multiprocessors , 1990 .

[5]  Kai Hwang,et al.  Critical issues in mapping neural networks on message-passing multicomputers , 1988, ISCA '88.

[6]  Philip D. Wasserman,et al.  Neural computing - theory and practice , 1989 .

[7]  Mohan Kumar,et al.  Extended Hypercube: A Hierarchical Interconnection Network of Hypercubes , 1992, IEEE Trans. Parallel Distributed Syst..

[8]  Richard P. Lippmann,et al.  An introduction to computing with neural nets , 1987 .

[9]  D J Evans,et al.  Parallel processing , 1986 .

[10]  Lalit M. Patnaik,et al.  Modelling Neural Networks , 1989 .

[11]  Lalit M. Patnaik,et al.  Efficient implementation of bidirectional associative memories on the extended hypercube , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[12]  Kunihiko Fukushima,et al.  Neocognitron: A new algorithm for pattern recognition tolerant of deformations and shifts in position , 1982, Pattern Recognit..

[13]  Jenq-Neng Hwang,et al.  A Unified Systolic Architecture for Artificial Neural Networks , 1989, J. Parallel Distributed Comput..

[14]  Kai Hwang,et al.  Hypernet: A Communication-Efficient Architecture for Constructing Massively Parallel Computers , 1987, IEEE Transactions on Computers.

[15]  D. Roweth,et al.  Implementing Neural Network Models on Parallel Computers , 1987, Comput. J..

[16]  Kai Hwang,et al.  Mapping Neural Networks onto Message-Passing Multicomputers , 1989, J. Parallel Distributed Comput..

[17]  Jim Bailey,et al.  A VLSI interconnect structure for neural networks , 1988 .

[18]  D. S. Touretzky,et al.  Neural network simulation at Warp speed: how we got 17 million connections per second , 1988, IEEE 1988 International Conference on Neural Networks.