Classical conditioning with pulsed integrated neural networks: circuits and system

In this paper, we investigate on-chip learning for pulsed, integrated neural networks. We discuss the implementational problems the technology imposes on learning systems, and we find that a biologically inspired approach using simple circuit structures is most likely to bring success. We develop a suitable learning algorithm-a continuous-time version of a temporal differential Hebbian learning algorithm for pulsed neural systems with nonlinear synapses-as well as circuits for the electronic implementation. Measurements from an experimental CMOS chip are presented. Finally, we use our test chip to solve simple classical conditioning tasks, thus verifying the design methodologies put forward in the paper.

[1]  J. J. Paulos,et al.  On-chip learning in the analog domain with limited precision circuits , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[2]  Clark S. Lindsey,et al.  Review of hardware neural networks: A User's perspective , 1994 .

[3]  Torsten Lehmann,et al.  Hardware Learning in Analogue VLSI Neural Networks , 1995 .

[4]  Daniel J. Amit,et al.  Learning in Neural Networks with Material Synapses , 1994, Neural Computation.

[5]  Ronald S. Gyurcsik,et al.  Building blocks for a temperature-compensated analog VLSI neural network with on-chip learning , 1994, Proceedings of IEEE International Symposium on Circuits and Systems - ISCAS '94.

[6]  Alan F. Murray.  Applications of Neural Networks , 1994 .

[7]  Torsten Lehmann,et al.  Mixed analog/digital matrix-vector multiplier for neural network synapses , 1996 .

[8]  David P. M. Northmore,et al.  Switched-capacitor neuromorphs with wide-range variable dynamics , 1995, IEEE Trans. Neural Networks.

[9]  Torsten Lehmann,et al.  On-chip learning in pulsed silicon neural networks , 1997, Proceedings of 1997 IEEE International Symposium on Circuits and Systems. Circuits and Systems in the Information Age ISCAS '97.

[10]  Takashi Morie,et al.  An all-analog expandable neural network LSI with on-chip backpropagation learning , 1994, IEEE J. Solid State Circuits.

[11]  D. Levine.  Introduction to Neural and Cognitive Modeling , 2018 .

[12]  Mona E. Zaghloul,et al.  Silicon Implementation of Pulse Coded Neural Networks , 1994 .

[13]  L. Merlat,et al.  A Hindmarsh and Rose-based electronic burster , 1996, Proceedings of Fifth International Conference on Microelectronics for Neural Networks.

[14]  E. Vittoz,et al.  Analog Storage of Adjustable Synaptic Weights , 1991 .

[15]  Eric A. Vittoz,et al.  Analog VLSI signal processing: Why, where, and how? , 1994, J. VLSI Signal Process..

[16]  Marwan A. Jabri,et al.  Weight perturbation: an optimal architecture and learning technique for analog VLSI feedforward and recurrent multilayer networks , 1992, IEEE Trans. Neural Networks.

[17]  Tetsuro Itakura,et al.  Neuro chips with on-chip back-propagation and/or Hebbian learning , 1992 .

[18]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[19]  A. Klopf.  A neuronal model of classical conditioning , 1988 .

[20]  Mark A. Gluck,et al.  Learning with Temporal Derivatives in Pulse-Coded Neuronal Systems , 1988, NIPS.

[21]  Alan F. Murray,et al.  Pulse-stream VLSI neural networks mixing analog and digital techniques , 1991, IEEE Trans. Neural Networks.

[22]  H. Shinohara,et al.  A refreshable analog VLSI neural network chip with 400 neurons and 40 K synapses , 1992 .

[23]  James S. Morgan,et al.  A drive-reinforcement neural network model of simple instrumental conditioning , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[24]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[25]  T. Lehmann.  Teaching pulsed integrated neural systems: a psychobiological approach , 1996, Proceedings of Fifth International Conference on Microelectronics for Neural Networks.

[26]  Davide Badoni,et al.  Electronic implementation of an analogue attractor neural network with stochastic learning , 1995 .

[27]  Ulrich Ramacher,et al.  Recent Developments In Neurodynamics And Their Impact On The Design Of Neuro-Chips , 1993, Int. J. Neural Syst..

[28]  R. Palmer,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[29]  Davide Badoni,et al.  LANN27: an electronic implementation of an analog attractor neural network with stochastic learning , 1995, SPIE Defense + Commercial Sensing.