SoC FPAA Hardware Implementation of a VMM+WTA Embedded Learning Classifier

This paper focuses on the circuit aspects required for an on-chip, on-line system on chip large-scale field-programmable analog array learning for vector-matrix multiplier (VMM) + winner-take-all (WTA) classifier structure. We start by describing the VMM+WTA classifier structure, and then show techniques required to handle device mismatch. The approach is initially explained using a VMM+WTA as a two-input XOR classifier structure. The approach requires considering the entire mixed-mode system, including the analog classifier data path, control circuitry for weight updates, and digital algorithm for computing digital weight updates and resulting floating-gate programming during the algorithm.

[1]  Robert A. Legenstein,et al.  Pattern representation and recognition with accelerated analog neuromorphic systems , 2017, 2017 IEEE International Symposium on Circuits and Systems (ISCAS).

[2]  Naveen Verma,et al.  A Low-Power Processor With Configurable Embedded Machine-Learning Accelerators for High-Order and Adaptive Analysis of Medical-Sensor Signals , 2013, IEEE Journal of Solid-State Circuits.

[3]  Sihwan Kim,et al.  Integrated Floating-Gate Programming Environment for System-Level ICs , 2016, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[4]  Jennifer Hasler,et al.  VMM + WTA Embedded Classifiers Learning Algorithm Implementable on SoC FPAA Devices , 2018, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[5]  M. A. Jabri,et al.  A low power trainable analogue neural network classifier chip , 1993, Proceedings of IEEE Custom Integrated Circuits Conference - CICC '93.

[6]  Wolfgang Maass,et al.  On the Computational Power of Winner-Take-All , 2000, Neural Computation.

[7]  John Lazzaro,et al.  Winner-Take-All Networks of O(N) Complexity , 1988, NIPS.

[8]  David Blaauw,et al.  A Fixed-Point Neural Network Architecture for Speech Applications on Resource Constrained Hardware , 2016, Journal of Signal Processing Systems.

[9]  Ethem Alpaydin,et al.  Introduction to machine learning , 2004, Adaptive computation and machine learning.

[10]  C PlattJ,et al.  A Neural Network Classifier for the I1000 OCR Chip. , 1996 .

[11]  Sihwan Kim,et al.  Scaling Floating-Gate Devices Predicting Behavior for Programmable and Configurable Circuits and Systems , 2016 .

[12]  Jennifer Hasler,et al.  Modeling, simulation and implementation of circuit elements in an open-source tool set on the FPAA , 2017 .

[13]  Gert Cauwenberghs,et al.  An analog VLSI recurrent neural network learning a continuous-time trajectory , 1996, IEEE Trans. Neural Networks.

[14]  Jennifer Hasler,et al.  Vector-Matrix Multiply and Winner-Take-All as an Analog Classifier , 2014, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[15]  Jennifer Hasler,et al.  Opportunities in physical computing driven by analog realization , 2016, 2016 IEEE International Conference on Rebooting Computing (ICRC).

[16]  Simha Sethumadhavan,et al.  Energy-Efficient Hybrid Analog/Digital Approximate Computation in Continuous Time , 2016, IEEE Journal of Solid-State Circuits.

[17]  Zhengya Zhang,et al.  A 6.67mW sparse coding ASIC enabling on-chip learning and inference , 2014, 2014 Symposium on VLSI Circuits Digest of Technical Papers.

[18]  Giacomo Indiveri,et al.  A reconfigurable on-line learning spiking neuromorphic processor comprising 256 neurons and 128K synapses , 2015, Front. Neurosci..

[19]  Gert Cauwenberghs,et al.  Neuromorphic learning VLSI systems: a survey , 1998 .

[20]  Jennifer Hasler,et al.  Reconfigurable analog classifier for knee-joint rehabilitation , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[21]  Fabrice Seguin,et al.  Ultra-Low-Energy Mixed-Signal IC Implementing Encoded Neural Networks , 2016, IEEE Transactions on Circuits and Systems I: Regular Papers.

[22]  K. Ming Leung,et al.  Learning Vector Quantization , 2017, Encyclopedia of Machine Learning and Data Mining.

[23]  Jennifer Hasler,et al.  Low power speech detector on a FPAA , 2017, 2017 IEEE International Symposium on Circuits and Systems (ISCAS).

[24]  Marwan A. Jabri,et al.  Weight perturbation: an optimal architecture and learning technique for analog VLSI feedforward and recurrent multilayer networks , 1992, IEEE Trans. Neural Networks.

[25]  K. Hirotsu,et al.  An analog neural network chip with random weight change learning algorithm , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[26]  Geoffrey E. Hinton,et al.  Acoustic Modeling Using Deep Belief Networks , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[27]  David W. Graham,et al.  Reconfiguration costs in analog sensor interfaces for wireless sensing applications , 2013, 2013 IEEE 56th International Midwest Symposium on Circuits and Systems (MWSCAS).

[28]  David V. Anderson,et al.  A floating-gate vector-quantizer , 2002, The 2002 45th Midwest Symposium on Circuits and Systems, 2002. MWSCAS-2002..

[29]  David V. Anderson,et al.  An analog programmable multi-dimensional radial basis function based classifier , 2007, 2007 IFIP International Conference on Very Large Scale Integration.

[30]  Venkatesh Srinivasan,et al.  A 531 nW/MHz, 128/spl times/32 current-mode programmable analog vector-matrix multiplier with over two decades of linearity , 2004, Proceedings of the IEEE 2004 Custom Integrated Circuits Conference (IEEE Cat. No.04CH37571).

[31]  Jennifer Hasler,et al.  A proof-of-concept classifier for acoustic signals from the knee joint on a FPAA , 2016, 2016 IEEE SENSORS.

[32]  Paul E. Hasler,et al.  A Highly Dense, Low Power, Programmable Analog Vector-Matrix Multiplier: The FPAA Implementation , 2011, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[33]  Jussi H. Poikonen,et al.  A mixed-mode array computing architecture for online dictionary learning , 2017, 2017 IEEE International Symposium on Circuits and Systems (ISCAS).

[34]  Sihwan Kim,et al.  Calibration of Floating-Gate SoC FPAA System , 2017, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[35]  Sihwan Kim,et al.  A Programmable and Configurable Mixed-Mode FPAA SoC , 2016, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[36]  Tadashi Shibata,et al.  A Neuron-MOS-Based VLSI Implementation of Pulse-Coupled Neural Networks for Image Feature Generation , 2010, IEEE Transactions on Circuits and Systems I: Regular Papers.

[37]  Steven R. Young,et al.  A 1 TOPS/W Analog Deep Machine-Learning Engine With Floating-Gate Storage in 0.13 µm CMOS , 2014, IEEE Journal of Solid-State Circuits.

[38]  Hong Yan,et al.  Handwritten digit recognition by adaptive-subspace self-organizing map (ASSOM) , 1999, IEEE Trans. Neural Networks.

[39]  Jennifer Hasler,et al.  Tuning of Multiple Parameters With a BIST System , 2017, IEEE Transactions on Circuits and Systems I: Regular Papers.

[40]  David V. Anderson,et al.  An Analog Programmable Multidimensional Radial Basis Function Based Classifier , 2007, IEEE Transactions on Circuits and Systems I: Regular Papers.

[41]  Jennifer Hasler.  Energy Constraints for Building Large-Scale Systems , 2016 .

[42]  Nils J. Nilsson,et al.  Introduction to Machine Learning , 2020, Machine Learning for iOS Developers.

[43]  J. Kwong,et al.  An Energy-Efficient Biomedical Signal Processing Platform , 2010, IEEE Journal of Solid-State Circuits.

[44]  Paul E. Hasler,et al.  An analog floating-gate node for Supervised learning , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.