An analog floating-gate node for Supervised learning

We present an improved analog floating-gate pFET synapse that implements a supervised learning algorithm similar to the least mean square (LMS) learning rule. Weight decay plays a key role in several learning rules; this floating-gate synapse exhibits this behavior. We examine implications of the weight decay appearing in the correlation learning rule realized in the floating-gate synapse and provide experimental data characterizing the synapse and its performance in one-input and two-input LMS networks. Analog floating-gate synapses will enable larger-scale, on-chip learning networks than previously possible.

[1]  Peter M. Clarkson,et al.  Optimal and Adaptive Signal Processing , 1993 .

[2]  S. Tam,et al.  An electrically trainable artificial neural network (ETANN) with 10240 'floating gate' synapses , 1990, International 1989 Joint Conference on Neural Networks.

[3]  David Hsu,et al.  Learning Spike-Based Correlations and Conditional Probabilities in Silicon , 2001, NIPS.

[4]  David V. Anderson,et al.  Programmable and adaptive analog filters using arrays of floating-gate circuits , 2001, Proceedings 2001 Conference on Advanced Research in VLSI. ARVLSI 2001.

[5]  Paul Hasler,et al.  A continuous time synapse employing a refreshable multilevel memory , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[6]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[7]  Paul E. Hasler,et al.  Single Transistor Learning Synapses , 1994, NIPS.

[8]  Paul E. Hasler,et al.  Single transistor learning synapse with long term storage , 1995, Proceedings of ISCAS'95 - International Symposium on Circuits and Systems.

[9]  Paul Hasler,et al.  Improved correlation learning rule in continuously adapting floating-gate arrays using logarithmic pre-distortion of input and learning signals , 2002, 2002 IEEE International Symposium on Circuits and Systems. Proceedings (Cat. No.02CH37353).

[10]  Mohammed Ismail,et al.  High Frequency CMOS Transconductors , 1993 .

[11]  Paul Hasler,et al.  Offset removal using floating-gate circuits for mixed-signal systems , 2003, Southwest Symposium on Mixed-Signal Design, 2003..

[12]  Paul Hasler,et al.  Modeling Hot-Electron Injection in pFET's , 2003 .

[13]  Andreas G. Andreou,et al.  Current-mode subthreshold MOS implementation of the Herault-Jutten autoadaptive network , 1992 .

[14]  Bernabé Linares-Barranco,et al.  On the design and characterization of femtoampere current-mode circuits , 2003, IEEE J. Solid State Circuits.

[15]  Gert Cauwenberghs,et al.  Analysis and verification of an analog VLSI incremental outer-product learning system , 1992, IEEE Trans. Neural Networks.

[16]  Venkatesh Srinivasan,et al.  A 531 nW/MHz, 128/spl times/32 current-mode programmable analog vector-matrix multiplier with over two decades of linearity , 2004, Proceedings of the IEEE 2004 Custom Integrated Circuits Conference (IEEE Cat. No.04CH37571).

[17]  Z. Czarnul.  Novel MOS resistive circuit for synthesis of fully integrated continuous-time filters , 1986 .

[18]  Alan F. Murray,et al.  Floating gate memories for pulse-stream neural networks , 1997 .

[19]  Paul Hasler,et al.  Multiple-input translinear element networks , 2001 .

[20]  Phillip E Allen,et al.  CMOS Analog Circuit Design , 1987 .

[21]  Gert Cauwenberghs,et al.  Fault-tolerant dynamic multilevel storage in analog VLSI , 1994 .

[22]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[23]  Philipp Häfliger,et al.  Floating gate analog memory for parameter and variable storage in a learning silicon neuron , 1999, ISCAS'99. Proceedings of the 1999 IEEE International Symposium on Circuits and Systems VLSI (Cat. No.99CH36349).

[24]  Carver A. Mead,et al.  Neuromorphic electronic systems , 1990, Proc. IEEE.

[25]  Tor Sverre Lande,et al.  Overview of floating-gate devices, circuits, and systems , 2001 .

[26]  Paul E. Hasler,et al.  Correlation learning rule in floating-gate pFET synapses , 1999, ISCAS'99. Proceedings of the 1999 IEEE International Symposium on Circuits and Systems VLSI (Cat. No.99CH36349).

[27]  James R. Mann,et al.  An Analog Self-Organizing Neural Network Chip , 1988, NIPS.

[28]  Paul Hasler,et al.  Continuous-time feedback in floating-gate MOS circuits , 1998, ISCAS '98. Proceedings of the 1998 IEEE International Symposium on Circuits and Systems (Cat. No.98CH36187).

[29]  J. Mann,et al.  A self-organizing neural net chip , 1988, Proceedings of the IEEE 1988 Custom Integrated Circuits Conference.

[30]  B. A. Minch,et al.  Translinear circuits using subthreshold floating-gate MOS transistors , 1996 .

[31]  Carver Mead,et al.  Analog VLSI and neural systems , 1989 .