Digital Neural Network Implementations

This chapter gives an overview of existing digital VLSI implementations and discusses techniques for implementing high performance, high capacity digital neural nets. It presents a set of techniques for estimating chip area, performance, and power consumption in the early stages of design to facilitate architectural exploration. It shows how technology scaling rules can be included in the estimation process. It presents a set of basic building blocks useful in implementing digital networks. It then uses the estimation techniques to predict capacity and performance of a variety of digital architectures. Finally, it discusses implementation strategies for very large networks.

[1]  Michael J. Flynn,et al.  Introduction to Arithmetic for Digital Systems Designers , 1995 .

[2]  E. Capaldi,et al.  The organization of behavior. , 1992, Journal of applied behavior analysis.

[3]  Stuart Haber,et al.  A VLSI-efficient technique for generating multiple uncorrelated noise sources and its application to stochastic neural networks , 1991 .

[4]  Y. Tamura,et al.  A BiCMOS analog neural network with dynamically updated weights , 1992, 1990 37th IEEE International Conference on Solid-State Circuits.

[5]  Carver A. Mead,et al.  Neuromorphic electronic systems , 1990, Proc. IEEE.

[6]  Max Stanford Tomlinson,et al.  A digital neural network architecture for VLSI , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[7]  Gabriele Saucier,et al.  Silicon compiler for neuro-ASICs , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[8]  D. Hammerstrom,et al.  A VLSI architecture for high-performance, low-cost, on-chip learning , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[9]  Amnon Yariv,et al.  A CCD based neural network integrated circuit with 64K analog programmable synapses , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[10]  M. Yagyu,et al.  Design, fabrication and evaluation of a 5-inch wafer scale neural network LSI composed on 576 digital neurons , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[11]  Bing J. Sheu,et al.  VLSI image processor using analog programmable synapses and neurons , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[12]  David E. van den Bout,et al.  The VLSI implementation of STONN , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[13]  T. Yamada,et al.  A self-learning neural network chip with 125 neurons and 10 K self-organization synapses , 1990, Digest of Technical Papers., 1990 Symposium on VLSI Circuits.

[14]  Kazuo Yano,et al.  A 3.8-ns CMOS 16*16-b multiplier using complementary pass-transistor logic , 1990 .

[15]  R.R. Johnson.  Multichip modules: next-generation packages , 1990, IEEE Spectrum.

[16]  M.A. Horowitz,et al.  A single-chip, functional tester for VLSI circuits , 1990, 1990 37th IEEE International Conference on Solid-State Circuits.

[17]  A. Chiang,et al.  A programmable CCD signal processor , 1990, 1990 37th IEEE International Conference on Solid-State Circuits.

[18]  H.P. Graf,et al.  A reconfigurable CMOS neural network , 1990, 1990 37th IEEE International Conference on Solid-State Circuits.

[19]  秋山 泰.  The Gaussian machine : a stochastic, continuous neural network model , 1990 .

[20]  J. I. Raffel,et al.  A generic architecture for wafer-scale neuromorphic systems , 1990 .

[21]  M. Zubair,et al.  Systolic implementation of neural networks , 1989, Proceedings 1989 IEEE International Conference on Computer Design: VLSI in Computers and Processors.

[22]  Mark R Santoro.  Design and Clocking of VLSI Multipliers , 1989 .

[23]  S. Y. Kung,et al.  Digital VLSI architectures for neural networks , 1989, IEEE International Symposium on Circuits and Systems,.

[24]  Mark Horowitz,et al.  SPIM: a pipelined 64*64-bit iterative multiplier , 1989 .

[25]  Geoffrey E. Hinton.  Deterministic Boltzmann Learning Performs Steepest Descent in Weight-Space , 1989, Neural Computation.

[26]  Y. Hirai,et al.  A digital neuro-chip with unlimited connectability for large scale neural networks , 1989, International 1989 Joint Conference on Neural Networks.

[27]  D. E. Van den Bout,et al.  TInMANN: the integer Markovian artificial neural network , 1989, International 1989 Joint Conference on Neural Networks.

[28]  T. Watanabe,et al.  Neural network simulation on a massively parallel cellular array processor: AAP-2 , 1989, International 1989 Joint Conference on Neural Networks.

[29]  A. Masaki,et al.  A wafer scale integration neural network utilizing completely digital circuits , 1989, International 1989 Joint Conference on Neural Networks.

[30]  Carsten Peterson,et al.  Explorations of the mean field theory learning algorithm , 1989, Neural Networks.

[31]  A. Thakoor,et al.  Design of parallel hardware neural network systems from custom analog VLSI 'building block' chips , 1989, International 1989 Joint Conference on Neural Networks.

[32]  M. Duranton,et al.  Learning on VLSI: a general purpose digital neurochip , 1989, International 1989 Joint Conference on Neural Networks.

[33]  Robert B. Allen,et al.  Performance of a Stochastic Learning Microchip , 1990, NIPS.

[34]  D. S. Touretzky,et al.  Neural network simulation at Warp speed: how we got 17 million connections per second , 1988, IEEE 1988 International Conference on Neural Networks.

[35]  J. Bailey,et al.  Why VLSI implementations of associative VLCNs require connection multiplexing , 1988, IEEE 1988 International Conference on Neural Networks.

[36]  Alan F. Murray,et al.  Asynchronous VLSI neural networks using pulse-stream arithmetic , 1988 .

[37]  Wei Hwang,et al.  High-speed sensing scheme for CMOS DRAMs , 1988 .

[38]  Terrence J. Sejnowski,et al.  NETtalk: a parallel network that learns to read aloud , 1988 .

[39]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[40]  M. Horowitz,et al.  A Pipelined 64x64b Iterative Array Multiplier , 1988, 1988 IEEE International Solid-State Circuits Conference, 1988 ISSCC. Digest of Technical Papers.

[41]  Jenq-Neng Hwang,et al.  Ring systolic designs for artificial neural nets , 1988, Neural Networks.

[42]  Dan Hammerstrom,et al.  An interconnect structure for wafer scale neurocomputers , 1988, Neural Networks.

[43]  Guy E. Blelloch,et al.  Network Learning on the Connection Machine , 1987, IJCAI.

[44]  M. Aoki,et al.  An experimental large capacity semiconductor file memory using 16-levels/cell storage , 1987, 1987 Symposium on VLSI Circuits.

[45]  R. S. Withers,et al.  An artificial neural network integrated circuit based on MNOS/CCD principles , 1987 .

[46]  Lawrence D. Jackel,et al.  VLSI implementation of a neural network memory with several hundreds of neurons , 1987 .

[47]  N. Takagi,et al.  A high-speed multiplier using a redundant binary adder tree , 1987 .

[48]  A. P. Thakoor,et al.  A neurocomputer based on an analog-digital hybrid architecture , 1987 .

[49]  Massimo A. Sivilotti,et al.  Real-time visual computations using analog CMOS processing arrays , 1987 .

[50]  K. Shimohigashi,et al.  A 16-levels/cell dynamic memory , 1985, 1985 IEEE International Solid-State Circuits Conference. Digest of Technical Papers.

[51]  Kamran Eshraghian,et al.  Principles of CMOS VLSI Design: A Systems Perspective , 1985 .

[52]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[53]  W. J. Poppelbaum,et al.  Stochastic and deterministic averaging processors , 1981 .

[54]  G.W. Taylor,et al.  Leakage studies in high-density dynamic MOS memory devices , 1979, IEEE Transactions on Electron Devices.

[55]  Lynn Conway,et al.  Introduction to VLSI systems , 1978 .

[56]  D.A. Hodges,et al.  Multilevel random-access memory using one transistor per cell , 1976, IEEE Journal of Solid-State Circuits.

[57]  David A. Patterson,et al.  Computer Architecture: A Quantitative Approach , 1969 .

[58]  Brian R. Gaines,et al.  Stochastic computing , 1967, AFIPS '67 (Spring).

[59]  A. Avizeinis,et al.  Signed Digit Number Representations for Fast Parallel Arithmetic , 1961 .