On Real-Time AER 2-D Convolutions Hardware for Neuromorphic Spike-Based Cortical Processing

In this paper, a chip that performs real-time image convolutions with programmable kernels of arbitrary shape is presented. The chip is a first experimental prototype of reduced size to validate the implemented circuits and system level techniques. The convolution processing is based on the address-event-representation (AER) technique, which is a spike-based biologically inspired image and video representation technique that favors communication bandwidth for pixels with more information. As a first test prototype, a pixel array of 16times16 has been implemented with programmable kernel size of up to 16times16. The chip has been fabricated in a standard 0.35 mum complimentary metal-oxide-semiconductor (CMOS) process. The technique also allows to process larger size images by assembling 2D arrays of such chips. Pixel operation exploits low-power mixed analog-digital circuit techniques. Because of the low currents involved (down to nanoamperes or even picoamperes), an important amount of pixel area is devoted to mismatch calibration. The rest of the chip uses digital circuit techniques, both synchronous and asynchronous. The fabricated chip has been thoroughly tested, both at the pixel level and at the system level. Specific computer interfaces have been developed for generating AER streams from conventional computers and feeding them as inputs to the convolution chip, and for grabbing AER streams coming out of the convolution chip and storing and analyzing them on computers. Extensive experimental results are provided. At the end of this paper, we provide discussions and results on scaling up the approach for larger pixel arrays and multilayer cortical AER systems.

[1]  Bernabé Linares-Barranco,et al.  A Calibration Technique for Very Low Current and Compact Tunable Neuromorphic Cells: Application to 5-bit 20-nA DACs , 2008, IEEE Transactions on Circuits and Systems II: Express Briefs.

[2]  Denis Fize,et al.  Speed of processing in the human visual system , 1996, Nature.

[3]  John Wawrzynek,et al.  A multi-sender asynchronous extension to the AER protocol , 1995, Proceedings Sixteenth Conference on Advanced Research in VLSI.

[4]  Viktor Öwall,et al.  A Custom Image Convolution DSP with a Sustained Calculation Capacity of >1 GMAC/s and Low I/O Bandwidth , 1999, J. VLSI Signal Process..

[5]  Neil Weste,et al.  Principles of CMOS VLSI Design , 1985 .

[6]  Antonio Gentile,et al.  Portable video supercomputing , 2004, IEEE Transactions on Computers.

[7]  Jihong Kim,et al.  A low-power image convolution algorithm for variable voltage processors , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[8]  Bertram E. Shi,et al.  Neuromorphic implementation of orientation hypercolumns , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[9]  Kwabena Boahen,et al.  A throughput-on-demand address-event transmitter for neuromorphic chips , 1999, Proceedings 20th Anniversary Conference on Advanced Research in VLSI.

[10]  Bernabé Linares-Barranco,et al.  A Neuromorphic Cortical-Layer Microchip for Spike-Based Event Processing Vision Systems , 2006, IEEE Transactions on Circuits and Systems I: Regular Papers.

[11]  Rahul Sarpeshkar,et al.  Pulse-Based Analog VLSI Velocity Sensors , 1997 .

[12]  Christian Enz,et al.  CMOS low-power analog circuit design , 1996, Emerging Technologies: Designing Low Power Digital Systems.

[13]  Bernabé Linares-Barranco,et al.  High-speed character recognition system based on a complex hierarchical AER architecture , 2008, 2008 IEEE International Symposium on Circuits and Systems.

[14]  Bernabé Linares-Barranco,et al.  On the design and characterization of femtoampere current-mode circuits , 2003, IEEE J. Solid State Circuits.

[15]  Gert Cauwenberghs,et al.  Probabilistic synaptic weighting in a reconfigurable network of VLSI integrate-and-fire neurons , 2001, Neural Networks.

[16]  Kwabena Boahen.  Retinomorphic vision systems , 1996, Proceedings of Fifth International Conference on Microelectronics for Neural Networks.

[17]  E. I. El-Masry,et al.  Clock feedthrough analysis and cancellation in current sample/hold circuits , 1994 .

[18]  Peter Tino,et al.  IEEE Transactions on Neural Networks , 2009 .

[19]  T. Sacktor.  The Synaptic Organization of the Brain (3rd Ed.) , 1991 .

[20]  John Wawrzynek,et al.  Silicon Auditory Processors as Computer Peripherals , 1992, NIPS.

[21]  Arnaud Delorme,et al.  Spike-based strategies for rapid processing , 2001, Neural Networks.

[22]  April S. Brown,et al.  SIMPil: an OE integrated SIMD architecture for focal plane processing applications , 1996, Proceedings of Massively Parallel Processing Using Optical Interconnections.

[23]  Jacques Gautrais,et al.  Rank order coding , 1998 .

[24]  Stephen Grossberg,et al.  Synthetic aperture radar processing by a multiple scale neural system for boundary and surface representation , 1995, Neural Networks.

[25]  Alain J. Martin.  Programming in VLSI: from communicating processes to delay-insensitive circuits , 1991 .

[26]  E. Culurciello,et al.  A biomorphic digital image sensor , 2003, IEEE J. Solid State Circuits.

[27]  Beat Fasel,et al.  Robust face analysis using convolutional neural networks , 2002, Object recognition supported by user interaction for service robots.

[28]  Massimo A. Sivilotti,et al.  Wiring considerations in analog VLSI systems, with application to field-programmable networks , 1992 .

[29]  Thierry M. Bernard,et al.  Making the most of 15kλ2 silicon area for a digital retina PE , 1998, Other Conferences.

[30]  Bernabé Linares-Barranco,et al.  Inter-spike-intervals analysis of AER Poisson-like generator hardware , 2007, Neurocomputing.

[31]  Kwabena Boahen,et al.  Point-to-point connectivity between neuromorphic chips using address events , 2000 .

[32]  Eugenio Culurciello,et al.  A comparative study of access topologies for chip-level address-event communication channels , 2003, IEEE Trans. Neural Networks.

[33]  Bernabé Linares-Barranco,et al.  A Spatial Contrast Retina With On-Chip Calibration for Neuromorphic Spike-Based AER Vision Systems , 2007, IEEE Transactions on Circuits and Systems I: Regular Papers.

[34]  Gert Cauwenberghs,et al.  Saliency-driven image acuity modulation on a reconfigurable silicon array of spiking neurons , 2004, NIPS 2004.

[35]  Eric A. Vittoz,et al.  A communication architecture tailored for analog VLSI artificial neural networks: intrinsic performance and limitations , 1994, IEEE Trans. Neural Networks.

[36]  KUNIHIKO FUKUSHIMA,et al.  Visual Feature Extraction by a Multilayered Network of Analog Threshold Elements , 1969, IEEE Trans. Syst. Sci. Cybern..

[37]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[38]  Kunihiko Fukushima,et al.  Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position , 1980, Biological Cybernetics.

[39]  Claus Nebauer,et al.  Evaluation of convolutional neural networks for visual recognition , 1998, IEEE Trans. Neural Networks.

[40]  Ralph Etienne-Cummings,et al.  A programmable focal-plane MIMD image processor chip , 2001, IEEE J. Solid State Circuits.

[41]  Yoshua Bengio,et al.  Convolutional networks for images, speech, and time series , 1998 .

[42]  Bernabé Linares-Barranco,et al.  On algorithmic rate-coded AER generation , 2006, IEEE Transactions on Neural Networks.

[43]  Tobi Delbrück,et al.  A 128 X 128 120db 30mw asynchronous vision sensor that responds to relative intensity change , 2006, 2006 IEEE International Solid State Circuits Conference - Digest of Technical Papers.

[44]  Tetsuya Yagi,et al.  A multichip aVLSI system emulating orientation selectivity of primary visual cortical cells , 2005, IEEE Transactions on Neural Networks.

[45]  Kwabena Boahen,et al.  Optic nerve signals in a neuromorphic chip I: Outer and inner retina models , 2004, IEEE Transactions on Biomedical Engineering.

[46]  Tobi Delbrück,et al.  Fully programmable bias current generator with 24 bit resolution per bias , 2006, 2006 IEEE International Symposium on Circuits and Systems.

[47]  K Fukushima,et al.  Handwritten alphanumeric character recognition by the neocognitron , 1991, IEEE Trans. Neural Networks.

[48]  K. Bult,et al.  An inherently linear and compact MOST-only current-division technique , 1992, 1992 IEEE International Solid-State Circuits Conference Digest of Technical Papers.

[49]  Bertram E. Shi,et al.  An ON-OFF orientation selective address event representation image transceiver chip , 2004, IEEE Transactions on Circuits and Systems I: Regular Papers.

[50]  G. Shepherd.  The Synaptic Organization of the Brain , 1979 .

[51]  Bernabé Linares-Barranco,et al.  A new charge-packet driven mismatch-calibrated integrate-and-fire neuron for processing positive and negative signals in AER based systems , 2004, 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512).

[52]  Alejandro Linares-Barranco,et al.  AER tools for communications and debugging , 2006, 2006 IEEE International Symposium on Circuits and Systems.

[53]  Andreas G. Andreou,et al.  AER image filtering architecture for vision-processing systems , 1999 .

[54]  Misha A. Mahowald,et al.  An Analog VLSI System for Stereoscopic Vision , 1994 .

[55]  Teresa Serrano-Gotarredona,et al.  Event generators for address event representation transmitters , 2005, SPIE Microtechnologies.

[56]  Christopher J. Bishop,et al.  Pulsed Neural Networks , 1998 .

[57]  Eric A. Vittoz,et al.  A communication scheme for analog VLSI perceptive systems , 1995 .

[58]  Andreas G. Andreou,et al.  A Programmable VLSI Filter Architecture for Application in Real-Time Vision Processing Systems , 2000, Int. J. Neural Syst..

[59]  Bernabé Linares-Barranco,et al.  A real-time clustering microchip neural engine , 1996, IEEE Trans. Very Large Scale Integr. Syst..

[60]  Bernabé Linares-Barranco,et al.  The Stochastic I-Pot: A Circuit Block for Programming Bias Currents , 2007, IEEE Transactions on Circuits and Systems II: Express Briefs.

[61]  T. Serrano-Gotarredona,et al.  CMOS transistor mismatch model valid from weak to strong inversion , 2003, ESSCIRC 2004 - 29th European Solid-State Circuits Conference (IEEE Cat. No.03EX705).

[62]  Eric A. Vittoz,et al.  An integrated cortical layer for orientation enhancement , 1997 .

[63]  Bernabé Linares-Barranco,et al.  Compact low-power calibration mini-DACs for neural arrays with programmable weights , 2003, IEEE Trans. Neural Networks.

[64]  Ah Chung Tsoi,et al.  Face recognition: a convolutional neural-network approach , 1997, IEEE Trans. Neural Networks.

[65]  Kwabena Boahen,et al.  Optic nerve signals in a neuromorphic chip II: testing and results , 2004, IEEE Transactions on Biomedical Engineering.

[66]  G. Geelen,et al.  An inherently linear and compact MOST-only current division technique , 1992 .