3-D wafer scale architectures for neural network computing

We introduce a class of massively parallel computer architectures which can be configured to efficiently handle a variety of neural network models. The underlying technology is three-dimensional wafer scale integration (3-D WSI), which provides an ideal medium to construct powerful, compact, and low power hardware tailored for neural network processing. A second generation prototype computer consisting of a 128*128 array of processors formed by stacking 8 CMOS wafers is nearing completion. The performance of this prototype is compared with enhanced architectures configured with special wafer types to accelerate neural network operations. The design of these specialized resources emphasizes the synergy between neural processing functions and the 3-D WSI architecture and packaging. Detailed microcode emulations are used to assess the impact of different algorithm/architecture modifications. Neural networks for cooperative vision integration and multilayer backpropagation are mapped onto various 3-D wafer stacks. Estimated performance for the vision integration network is 2.4 billion connections per second. For the backprop network training algorithm, the performance ranges from 1.1 billion connection updates per second (GCUPS) for a near-term 128*128 prototype up to 53.4 GCUPS for a future 512*512 machine with more extensive neural processing hardware enhancements. >

[1]  S. Grossberg,et al.  Pattern Recognition by Self-Organizing Neural Networks , 1991 .

[2]  C.A. Neugebauer,et al.  Future trends in wafer scale integration , 1986, Proceedings of the IEEE.

[3]  J. G. Nash,et al.  Redundancy for yield enhancement in the 3-D computer , 1989, [1989] Proceedings International Conference on Wafer Scale Integration.

[4]  James J. Clark,et al.  Data Fusion for Sensory Information Processing Systems , 1990 .

[5]  Scott T. Toborg.  A 3-D wafer scale architecture for early vision processing , 1990, [1990] Proceedings of the International Conference on Application Specific Array Processors.

[6]  M. L. Campbell,et al.  3D wafer stack neurocomputing , 1993, 1993 Proceedings Fifth Annual IEEE International Conference on Wafer Scale Integration.

[7]  Bertil Svensson,et al.  Using and Designing Massively Parallel Computers for Artificial Neural Neural Networks , 1992, J. Parallel Distributed Comput..

[8]  M. Little,et al.  Hierarchical fault tolerance for 3D microelectronics , 1990, 1990 Proceedings. International Conference on Wafer Scale Integration.

[9]  Ulrich Ramacher,et al.  SYNAPSE - A Neurocomputer that Synthesizes Neural Algorithms on a Parallel Systolic Engine , 1992, J. Parallel Distributed Comput..

[10]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[11]  Jan Grinberg,et al.  The 3-D Computer: An Integrated Stack of WSI Wafers , 1989 .

[12]  Tomaso Poggio,et al.  Computational vision and regularization theory , 1985, Nature.

[13]  M. J. Little,et al.  Reliability of the 3-D computer under stress of mechanical vibration and thermal cycling , 1989, [1989] Proceedings International Conference on Wafer Scale Integration.

[14]  Kai Hwang,et al.  Cooperative Vision Integration Through Data-Parallel Neural Computations , 1991, IEEE Trans. Computers.

[15]  Scott Thomas Toborg.  Cooperative vision integration: from neural network algorithms to wafer scale architectures , 1992 .

[16]  Kunihiko Fukushima,et al.  Neocognitron: A hierarchical neural network capable of visual pattern recognition , 1988, Neural Networks.