Backpropagation Multilayer Perceptron: A Modular Implementation

In this paper we present a VLSI systolic architecture for implementing the multilayer perceptron network. Both the retrieving phase and the backpropagation learning algorithm with a sigmoid nonlinearity are considered. The implementation is based upon a variable size ring systolic array which is highly regular thus favouring integration. This architecture can also be expanded in a parallel or cascade fashion to implement arbitrary size networks.

[1]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[2]  Philippe Hurat,et al.  A VLSI Systolic Array Dedicated to Hopfield Neural Network , 1989 .

[3]  José G. Delgado-Frias,et al.  VLSI for Artificial Intelligence , 1989 .

[4]  David J. Burr,et al.  Experiments on neural net recognition of spoken and written text , 1988, IEEE Trans. Acoust. Speech Signal Process..

[5]  Jenq-Neng Hwang,et al.  A systolic neural network architecture for hidden Markov models , 1989, IEEE Trans. Acoust. Speech Signal Process..

[6]  Jenq-Neng Hwang,et al.  A Unified Systolic Architecture for Artificial Neural Networks , 1989, J. Parallel Distributed Comput..

[7]  R. Hecht-Nielsen,et al.  Neurocomputing: picking the human brain , 1988, IEEE Spectrum.

[8]  A. Masaki,et al.  Neural networks in CMOS: a case study , 1990, IEEE Circuits and Devices Magazine.

[9]  Michel Weinfeld,et al.  A Fully Digital Integrated CMOS Hopfield Network Including the Learning Algorithm , 1989 .

[10]  D. J. Myers,et al.  Efficient implementation of piecewise linear activation function for digital VLSI neural networks , 1989 .

[11]  Marina C. Chen,et al.  A Design Methodology for Synthesizing Parallel Algorithms and Architectures , 1986, J. Parallel Distributed Comput..

[12]  Seungryoul Maeng,et al.  Parallel simulation of multilayered neural networks on distributed-memory multiprocessors , 1990 .

[13]  S. Y. King.  Parallel architectures for artificial neural nets , 1988, [1988] Proceedings. International Conference on Systolic Arrays.

[14]  Mateo Valero,et al.  Partitioning: An Essential Step in Mapping Algorithms Into Systolic Array Processors , 1987, Computer.

[15]  R. Pfeifer,et al.  Connectionism in Perspective , 1989 .

[16]  M. Valero-Garcia,et al.  Systematic hardware adaptation of systolic algorithms , 1989, ISCA '89.

[17]  J. Alspector,et al.  Neural-style microsystems that learn , 1989, IEEE Communications Magazine.

[18]  Michael J. Witbrock,et al.  An implementation of backpropagation learning on GF11, a large SIMD parallel computer , 1990, Parallel Comput..

[19]  J.C. Lupo.  Defense applications of neural networks , 1989, IEEE Communications Magazine.

[20]  Y. Suzuki,et al.  Digital systems for artificial neural networks , 1989, IEEE Circuits and Devices Magazine.

[21]  Dan I. Moldovan,et al.  Partitioning and Mapping Algorithms into Fixed Size Systolic Arrays , 1986, IEEE Transactions on Computers.

[22]  R. Lippmann,et al.  An introduction to computing with neural nets , 1987, IEEE ASSP Magazine.

[23]  G. Baudet,et al.  Pragmatic approach to systolic design , 1990 .

[24]  Klaus Schumacher,et al.  VLSI technologies for artificial neural networks , 1989, IEEE Micro.

[25]  Kunihiko Fukushima,et al.  Neocognitron: A hierarchical neural network capable of visual pattern recognition , 1988, Neural Networks.

[26]  S. Barro,et al.  A VLSI and WSI systolic architecture for multilayer feed-forward networks of arbitrary size , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[27]  Paul Hasler,et al.  A CMOS neural network for pattern association , 1989, IEEE Micro.

[28]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[29]  Jill P. Mesirov,et al.  The backpropagation algorithm on grid and hypercube architectures , 1990, Parallel Comput..

[30]  R. Ishii,et al.  Neural net implementation on single-chip digital signal processor , 1989, 15th Annual Conference of IEEE Industrial Electronics Society.