A Neuromorphic Cortical-Layer Microchip for Spike-Based Event Processing Vision Systems

We present a neuromorphic cortical-layer processing microchip for address event representation (AER) spike-based processing systems. The microchip computes 2-D convolutions of video information represented in AER format in real time. AER, as opposed to conventional frame-based video representation, describes visual information as a sequence of events or spikes in a way similar to biological brains. This format allows for fast information identification and processing, without waiting to process complete image frames. The neuromorphic cortical-layer processing microchip presented in this paper computes convolutions of programmable kernels over the AER visual input information flow. It not only computes convolutions but also allows for a programmable forgetting rate, which in turn allows for a bio-inspired coincidence detection processing. Kernels are programmable and can be of arbitrary shape and arbitrary size of up to 32 times 32 pixels. The convolution processor operates on a pixel array of size 32 times 32, but can process an input space of up to 128 times 128 pixels. Larger pixel arrays can be directly processed by tiling arrays of chips. The chip receives and generates data in AER format, which is asynchronous and digital. However, its internal operation is based on analog low-current circuit techniques. The paper describes the architecture of the chip and circuits used for the pixels, including calibration techniques to overcome mismatch. Extensive experimental results are provided, describing pixel operation and calibration, convolution processing with and without forgetting, and high-speed recognition experiments like discriminating rotating propellers of different shape rotating at speeds of up to 5000 revolutions per second

[1]  Ph. Hafliger.  Asynchronous event redirecting in bio-inspired communication , 2001, ICECS 2001. 8th IEEE International Conference on Electronics, Circuits and Systems (Cat. No.01EX483).

[2]  Tobi Delbrück,et al.  Improved ON/OFF temporally differentiating address-event imager , 2004, Proceedings of the 2004 11th IEEE International Conference on Electronics, Circuits and Systems, 2004. ICECS 2004..

[3]  André van Schaik,et al.  AER EAR: A Matched Silicon Cochlea Pair With Address Event Representation Interface , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[4]  W. D. Ross,et al.  Visual brain and visual perception: how does the cortex do perceptual grouping? , 1997, Trends in Neurosciences.

[5]  Kwabena Boahen,et al.  A burst-mode word-serial address-event link-I: transmitter design , 2004, IEEE Transactions on Circuits and Systems I: Regular Papers.

[6]  Yoshua Bengio,et al.  Convolutional networks for images, speech, and time series , 1998 .

[7]  Eric A. Vittoz,et al.  A communication scheme for analog VLSI perceptive systems , 1995 .

[8]  Mehdi Azadmehr.  A foveated aer imager chip , 2005 .

[9]  Katsuhiko Mori,et al.  Convolutional spiking neural network model for robust face detection , 2002, Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02..

[10]  Gert Cauwenberghs,et al.  Spatial acuity modulation of an address-event imager , 2004, Proceedings of the 2004 11th IEEE International Conference on Electronics, Circuits and Systems, 2004. ICECS 2004..

[11]  Stephen Grossberg,et al.  Synthetic aperture radar processing by a multiple scale neural system for boundary and surface representation , 1995, Neural Networks.

[12]  Bernabé Linares-Barranco,et al.  On algorithmic rate-coded AER generation , 2006, IEEE Transactions on Neural Networks.

[13]  Tobi Delbrück,et al.  A 128 X 128 120db 30mw asynchronous vision sensor that responds to relative intensity change , 2006, 2006 IEEE International Solid State Circuits Conference - Digest of Technical Papers.

[14]  Saeed Shiry Ghidary,et al.  Convolutional Neural Networks for Image Processing: An Application in Robot Vision , 2003, Australian Conference on Artificial Intelligence.

[15]  Tobi Delbrück,et al.  AER Building Blocks for Multi-Layer Multi-Chip Neuromorphic Vision Systems , 2005, NIPS.

[16]  Bertram E. Shi,et al.  An ON-OFF orientation selective address event representation image transceiver chip , 2004, IEEE Transactions on Circuits and Systems I: Regular Papers.

[17]  T. Serrano-Gotarredona,et al.  CMOS transistor mismatch model valid from weak to strong inversion , 2003, ESSCIRC 2004 - 29th European Solid-State Circuits Conference (IEEE Cat. No.03EX705).

[18]  F. Heitger,et al.  A 100×100 pixel silicon retina for gradient extraction with steering filter capabilities and temporal output coding , 2002, IEEE J. Solid State Circuits.

[19]  John Wawrzynek,et al.  A multi-sender asynchronous extension to the AER protocol , 1995, Proceedings Sixteenth Conference on Advanced Research in VLSI.

[20]  Mehdi Azadmehr,et al.  A foveated AER imager chip [address event representation] , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[21]  Kwabena Boahen.  A burst-mode word-serial address-event link-II: receiver design , 2004, IEEE Transactions on Circuits and Systems I: Regular Papers.

[22]  Eric A. Vittoz,et al.  An integrated cortical layer for orientation enhancement , 1997 .

[23]  Misha A. Mahowald,et al.  An Analog VLSI System for Stereoscopic Vision , 1994 .

[24]  Teresa Serrano-Gotarredona,et al.  Event generators for address event representation transmitters , 2005, SPIE Microtechnologies.

[25]  Bernabé Linares-Barranco,et al.  Compact low-power calibration mini-DACs for neural arrays with programmable weights , 2003, IEEE Trans. Neural Networks.

[26]  Massimo A. Sivilotti,et al.  Wiring considerations in analog VLSI systems, with application to field-programmable networks , 1992 .

[27]  T. Sacktor.  The Synaptic Organization of the Brain (3rd Ed.) , 1991 .

[28]  Eugenio Culurciello,et al.  Event-based imaging with active illumination in sensor networks , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[29]  Bernabé Linares-Barranco,et al.  A Spatial Contrast Retina With On-Chip Calibration for Neuromorphic Spike-Based AER Vision Systems , 2007, IEEE Transactions on Circuits and Systems I: Regular Papers.

[30]  John G. Harris,et al.  A time-to-first spike CMOS imager , 2004, 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512).

[31]  Massimo Barbaro,et al.  A 100/spl times/100 pixel silicon retina for gradient extraction with steering filter capabilities and temporal output coding , 2002 .

[32]  Kazuyuki Aihara,et al.  Dynamical Cell Assembly Hypothesis -- Theoretical Possibility of Spatio-temporal Coding in the Cortex , 1996, Neural Networks.

[33]  Jörg Kramer,et al.  An on/off transient imager with event-driven, asynchronous read-out , 2002, 2002 IEEE International Symposium on Circuits and Systems. Proceedings (Cat. No.02CH37353).

[34]  Claus Nebauer,et al.  Evaluation of convolutional neural networks for visual recognition , 1998, IEEE Trans. Neural Networks.

[35]  Kwabena Boahen.  A burst-mode word-serial address-event link-III: analysis and test results , 2004, IEEE Transactions on Circuits and Systems I: Regular Papers.

[36]  Bernabé Linares-Barranco,et al.  On Real-Time AER 2-D Convolutions Hardware for Neuromorphic Spike-Based Cortical Processing , 2008, IEEE Transactions on Neural Networks.

[37]  Eric A. Vittoz,et al.  A communication architecture tailored for analog VLSI artificial neural networks: intrinsic performance and limitations , 1994, IEEE Trans. Neural Networks.

[38]  José Luis Sevillano,et al.  PCI-AER interface for neuro-inspired spiking systems , 2006, 2006 IEEE International Symposium on Circuits and Systems.

[39]  Andreas G. Andreou,et al.  Asynchronous Communication of 2D Motion Information Using Winner-Takes-All Arbitration , 1997 .

[40]  Gert Cauwenberghs,et al.  Probabilistic synaptic weighting in a reconfigurable network of VLSI integrate-and-fire neurons , 2001, Neural Networks.

[41]  Amine Bermak,et al.  A low power CMOS imager based on time-to-first-spike encoding and fair AER , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[42]  Michael N. Shadlen,et al.  Noise, neural codes and cortical organization , 1994, Current Opinion in Neurobiology.

[43]  Bernabé Linares-Barranco,et al.  An arbitrary kernel convolution AER-transceiver chip for real-time image filtering , 2006, 2006 IEEE International Symposium on Circuits and Systems.

[44]  E. Culurciello,et al.  A biomorphic digital image sensor , 2003, IEEE J. Solid State Circuits.

[45]  Beat Fasel,et al.  Robust face analysis using convolutional neural networks , 2002, Object recognition supported by user interaction for service robots.

[46]  Kwabena Boahen,et al.  Point-to-point connectivity between neuromorphic chips using address events , 2000 .

[47]  Bernabé Linares-Barranco,et al.  On the design and characterization of femtoampere current-mode circuits , 2003, IEEE J. Solid State Circuits.

[48]  Gert Cauwenberghs,et al.  Silicon spike-based synaptic array and address-event transceiver , 2004, 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512).

[49]  Shih-Chii Liu,et al.  Spiking Inputs to a Winner-take-all Network , 2005, NIPS.

[50]  Philipp Häfliger,et al.  A time domain winner-take-all network of integrate-and-fire neurons , 2004, 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512).

[51]  Kwabena Boahen,et al.  Retinomorphic chips that see quadruple images , 1999, Proceedings of the Seventh International Conference on Microelectronics for Neural, Fuzzy and Bio-Inspired Systems.

[52]  Kwabena Boahen,et al.  Optic nerve signals in a neuromorphic chip I: Outer and inner retina models , 2004, IEEE Transactions on Biomedical Engineering.

[53]  G. Shepherd.  The Synaptic Organization of the Brain , 1979 .

[54]  Bernabé Linares-Barranco,et al.  A new charge-packet driven mismatch-calibrated integrate-and-fire neuron for processing positive and negative signals in AER based systems , 2004, 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512).

[55]  Erhan Ozalevli,et al.  Reconfigurable biologically inspired visual motion systems using modular neuromorphic VLSI chips , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[56]  John Wawrzynek,et al.  Silicon Auditory Processors as Computer Peripherals , 1992, NIPS.

[57]  Charles M. Higgins,et al.  A biologically inspired modular VLSI system for visual measurement of self-motion , 2002 .

[58]  Giacomo Indiveri,et al.  A reconfigurable neuromorphic VLSI multi-chip system applied to visual motion computation , 1999, Proceedings of the Seventh International Conference on Microelectronics for Neural, Fuzzy and Bio-Inspired Systems.

[59]  Kunihiko Fukushima,et al.  Neocognitron: A hierarchical neural network capable of visual pattern recognition , 1988, Neural Networks.

[60]  Kunihiko Fukushima,et al.  Neocognitron: A new algorithm for pattern recognition tolerant of deformations and shifts in position , 1982, Pattern Recognit..

[61]  Alejandro Linares-Barranco,et al.  AER tools for communications and debugging , 2006, 2006 IEEE International Symposium on Circuits and Systems.

[62]  Gert Cauwenberghs,et al.  An analog VLSI chip with asynchronous interface for auditory feature extraction , 1998 .

[63]  Andreas G. Andreou,et al.  AER image filtering architecture for vision-processing systems , 1999 .

[64]  Shih-Chii Liu,et al.  Feature competition in a spike-based winner-take-all VLSI network , 2006, 2006 IEEE International Symposium on Circuits and Systems.

[65]  Professor Dr. Guy A. Orban.  Neuronal Operations in the Visual Cortex , 1983, Studies of Brain Function.

[66]  Timothy K. Horiuchi,et al.  A VLSI model of the bat dorsal nucleus of the lateral lemniscus for azimuthal echolocation , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[67]  Kwabena Boahen,et al.  Communicating neuronal ensembles between neuromorphic chips , 1998 .

[68]  Gert Cauwenberghs,et al.  Saliency-driven image acuity modulation on a reconfigurable silicon array of spiking neurons , 2004, NIPS 2004.

[69]  KUNIHIKO FUKUSHIMA,et al.  Visual Feature Extraction by a Multilayered Network of Analog Threshold Elements , 1969, IEEE Trans. Syst. Sci. Cybern..

[70]  Tobi Delbrück,et al.  Orientation-Selective aVLSI Spiking Neurons , 2001, NIPS.

[71]  Gert Cauwenberghs,et al.  A real-time spike-domain sensory information processing system [image processing applications] , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[72]  Gert Cauwenberghs,et al.  Spike Timing-Dependent Plasticity in the Address Domain , 2002, NIPS.

[73]  Gert Cauwenberghs,et al.  An analog VLSI chip with asynchronous interface for auditory feature extraction , 1997, Proceedings of 1997 IEEE International Symposium on Circuits and Systems. Circuits and Systems in the Information Age ISCAS '97.

[74]  Bertram E. Shi,et al.  Neuromorphic implementation of orientation hypercolumns , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[75]  Giacomo Indiveri,et al.  An event-based VLSI network of integrate-and-fire neurons , 2004, 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512).

[76]  Pierre-Yves Burgi,et al.  A 128 /spl times/ 128 pixel 120 dB dynamic range vision sensor chip for image contrast and orientation extraction , 2003, 2003 IEEE International Solid-State Circuits Conference, 2003. Digest of Technical Papers. ISSCC..

[77]  Shih-Chii Liu,et al.  AER EAR: A Matched Silicon Cochlea Pair With Address Event Representation Interface , 2007, IEEE Trans. Circuits Syst. I Regul. Pap..

[78]  Philipp Häfliger.  Adaptive WTA With an Analog VLSI Neuromorphic Learning Chip , 2007, IEEE Transactions on Neural Networks.

[79]  Kunihiko Fukushima,et al.  Analysis of the process of visual pattern recognition by the neocognitron , 1989, Neural Networks.

[80]  Bernabé Linares-Barranco,et al.  Current Mode Techniques for Sub-pico-Ampere Circuit Design , 2004 .

[81]  M. Arias-Estrada,et al.  Motion vision sensor architecture with asynchronous self-signaling pixels , 1997, Proceedings Fourth IEEE International Workshop on Computer Architecture for Machine Perception. CAMP'97.

[82]  Stephen Grossberg,et al.  A neural network for enhancing boundaries and surfaces in synthetic aperture radar images , 1999, Neural Networks.