Bio-Inspired Asynchronous Pixel Event Tricolor Vision Sensor

This article investigates the potential of the first ever prototype of a vision sensor that combines tricolor stacked photo diodes with the bio-inspired asynchronous pixel event communication protocol known as Address Event Representation (AER). The stacked photo diodes are implemented in a 22 × 22 pixel array in a standard STM 90 nm CMOS process. Dynamic range is larger than 60 dB and pixels fill factor is 28%. The pixels employ either simple pulse frequency modulation (PFM) or a Time-to-First-Spike (TFS) mode. A heuristic linear combination of the chip's inherent pseudo colors serves to approximate RGB color representation. Furthermore, the sensor outputs can be processed to represent the radiation in the near infrared (NIR) band without employing external filters, and to color-encode direction of motion due to an asymmetry in the update rates of the different diode layers.

[1]  Misha Anne Mahowald,et al.  VLSI analogs of neuronal visual processing: a synthesis of form and function , 1992 .

[2]  Tobi Delbrück,et al.  Self-timed vertacolor dichromatic vision sensor for low power pattern detection , 2008, 2008 IEEE International Symposium on Circuits and Systems.

[3]  Bernabé Linares-Barranco,et al.  Fast Vision Through Frameless Event-Based Sensing and Convolutional Processing: Application to Texture Recognition , 2010, IEEE Transactions on Neural Networks.

[4]  Bernabé Linares-Barranco,et al.  A Five-Decade Dynamic-Range Ambient-Light-Independent Calibrated Signed-Spatial-Contrast AER Retina With 0.1-ms Latency and Optional Time-to-First-Spike Mode , 2010, IEEE Transactions on Circuits and Systems I: Regular Papers.

[5]  Philipp Häfliger,et al.  Live demonstration: A bio-inspired asynchronous pixel event tri-color vision sensor , 2012, 2012 IEEE International Symposium on Circuits and Systems.

[6]  J. Rabin.  The Retina: An Approachable Part of the Brain , 2013 .

[7]  E. Culurciello,et al.  A biomorphic digital image sensor , 2003, IEEE J. Solid State Circuits.

[8]  Tobi Delbrück,et al.  A 128$\times$ 128 120 dB 15 $\mu$s Latency Asynchronous Temporal Contrast Vision Sensor , 2008, IEEE Journal of Solid-State Circuits.

[9]  Daniel Matolin,et al.  A QVGA 143dB dynamic range asynchronous address-event PWM dynamic image sensor with lossless pixel-level video compression , 2010, 2010 IEEE International Solid-State Circuits Conference - (ISSCC).

[10]  Mehdi Azadmehr.  A foveated aer imager chip , 2005 .

[11]  Philipp Hafliger.  A spike based learning rule and its implementation in analog hardware , 2000 .

[12]  Robert Lange,et al.  3D time-of-flight distance measurement with custom solid-state image sensors in CMOS/CCD-technology , 2006 .

[13]  Andreas G. Andreou,et al.  A Contrast Sensitive Silicon Retina with Reciprocal Synapses , 1991, NIPS.

[14]  Tobi Delbrück,et al.  Event-Based Pixel Sensitive to Changes of Color and Brightness , 2011, IEEE Transactions on Circuits and Systems I: Regular Papers.

[15]  G. de Graaf,et al.  Smart Optical Sensor Systems in CMOS for Measuring Light Intensity and Colour , 1998 .

[16]  S. Sze.  Semiconductor Devices: Physics and Technology , 1985 .

[17]  Kwabena Boahen,et al.  Point-to-point connectivity between neuromorphic chips using address events , 2000 .

[18]  Yong Yan,et al.  Monitoring of oscillatory characteristics of pulverized coal flames through image processing and spectral analysis , 2006, IEEE Transactions on Instrumentation and Measurement.

[19]  Kwabena Boahen,et al.  The Retinomorphic Approach: Pixel-Parallel Adaptive Amplification, Filtering, and Quantization , 1997 .

[20]  Tobi Delbrück,et al.  Event-based color change pixel in standard CMOS , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[21]  Bernabé Linares-Barranco,et al.  An Event-Driven Multi-Kernel Convolution Processor Module for Event-Driven Vision Sensors , 2012, IEEE Journal of Solid-State Circuits.

[22]  Bernabé Linares-Barranco,et al.  A 3.6 $\mu$ s Latency Asynchronous Frame-Free Event-Driven Dynamic-Vision-Sensor , 2011, IEEE Journal of Solid-State Circuits.

[23]  R Van Rullen,et al.  Face processing using one spike per neurone. , 1998, Bio Systems.

[24]  Philipp Häfliger,et al.  Flame monitoring with an AER color vision sensor , 2013, 2013 IEEE International Symposium on Circuits and Systems (ISCAS2013).

[25]  Philipp Häfliger,et al.  Two color asynchronous event photo pixel , 2008, 2008 IEEE International Symposium on Circuits and Systems.

[26]  Philipp Hafliger,et al.  Bio-inspired asynchronous pixel event tri-color vision sensor , 2011 .

[27]  Daniel Matolin,et al.  A QVGA 143 dB Dynamic Range Frame-Free PWM Image Sensor With Lossless Pixel-Level Video Compression and Time-Domain CDS , 2011, IEEE Journal of Solid-State Circuits.

[28]  Misha Mahowald,et al.  A silicon model of early visual processing , 1993, Neural Networks.

[29]  Reinoud F. Wolffenbuttel,et al.  Colour-sensor system with a frequency output and an ISS or I2C bus interface , 1997 .

[30]  Tobi Delbrück,et al.  A 5 Meps $100 USB2.0 Address-Event Monitor-Sequencer Interface , 2007, 2007 IEEE International Symposium on Circuits and Systems.

[31]  Mehdi Azadmehr,et al.  A foveated AER imager chip [address event representation] , 2005, 2005 IEEE International Symposium on Circuits and Systems.