Activity-driven, event-based vision sensors

The four chips [1–4] presented in the special session on "Activity-driven, event-based vision sensors" quickly output compressed digital data in the form of events. These sensors reduce redundancy and latency and increase dynamic range compared with conventional imagers. The digital sensor output is easily interfaced to conventional digital post processing, where it reduces the latency and cost of post processing compared to imagers. The asynchronous data could spawn a new area of DSP that breaks from conventional Nyquist rate signal processing. This paper reviews the rationale and history of this event-based approach, introduces sensor functionalities, and gives an overview of the papers in this session. The paper concludes with a brief discussion on open questions.

[1]  Eric R. Fossum,et al.  CMOS image sensors: electronic camera-on-a-chip , 1997 .

[2]  Tobi Delbrück,et al.  A 128 X 128 120db 30mw asynchronous vision sensor that responds to relative intensity change , 2006, 2006 IEEE International Solid State Circuits Conference - Digest of Technical Papers.

[3]  Kwabena Boahen,et al.  A Retinomorphic Chip with Parallel Pathways: Encoding INCREASING, ON, DECREASING, and OFF Visual Signals , 2002 .

[4]  Misha Anne Mahowald,et al.  VLSI analogs of neuronal visual processing: a synthesis of form and function , 1992 .

[5]  Tobi Delbrück,et al.  Self-timed vertacolor dichromatic vision sensor for low power pattern detection , 2008, 2008 IEEE International Symposium on Circuits and Systems.

[6]  Eugenio Culurciello,et al.  A compact-pixel tri-mode vision sensor , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[7]  John G. Harris.  The changing roles of analog and digital signal processing in CMOS image sensors , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[8]  Daniel Matolin,et al.  A QVGA 143dB dynamic range asynchronous address-event PWM dynamic image sensor with lossless pixel-level video compression , 2010, 2010 IEEE International Solid-State Circuits Conference - (ISSCC).

[9]  Tobi Delbrück,et al.  CAVIAR: A 45k Neuron, 5M Synapse, 12G Connects/s AER Hardware Sensory–Processing– Learning–Actuating System for High-Speed Visual Object Recognition and Tracking , 2009, IEEE Transactions on Neural Networks.

[10]  F. Heitger,et al.  A 100×100 pixel silicon retina for gradient extraction with steering filter capabilities and temporal output coding , 2002, IEEE J. Solid State Circuits.

[11]  Mehdi Azadmehr.  A foveated aer imager chip , 2005 .

[12]  Philipp Häfliger,et al.  Two color asynchronous event photo pixel , 2008, 2008 IEEE International Symposium on Circuits and Systems.

[13]  Tobi Delbrück,et al.  Temporal contrast AER pixel with 0.3%-contrast event threshold , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[14]  Kwabena Boahen,et al.  Optic nerve signals in a neuromorphic chip II: testing and results , 2004, IEEE Transactions on Biomedical Engineering.

[15]  Massimo Barbaro,et al.  A 100/spl times/100 pixel silicon retina for gradient extraction with steering filter capabilities and temporal output coding , 2002 .

[16]  Kwabena Boahen.  A burst-mode word-serial address-event link-III: analysis and test results , 2004, IEEE Transactions on Circuits and Systems I: Regular Papers.

[17]  Heinrich Garn,et al.  Vehicle Counting with an Embedded Traffic Data System using an Optical Transient Sensor , 2007, 2007 IEEE Intelligent Transportation Systems Conference.

[18]  Gert Cauwenberghs,et al.  CMOS Camera With In-Pixel Temporal Change Detection and ADC , 2007, IEEE Journal of Solid-State Circuits.

[19]  R. Engelbrecht,et al.  DIGEST of TECHNICAL PAPERS , 1959 .

[20]  Bertram E. Shi,et al.  Neuromorphic implementation of orientation hypercolumns , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[21]  Tobi Delbrück,et al.  A Multichip Pulse-Based Neuromorphic Infrastructure and Its Application to a Model of Orientation Selectivity , 2007, IEEE Transactions on Circuits and Systems I: Regular Papers.

[22]  Daniel Matolin,et al.  High-DR frame-free PWM imaging with asynchronous AER intensity encoding and focal-plane temporal redundancy suppression , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[23]  Alessandro Mortara,et al.  A Pulsed Communication/Computation Framework for Analog VLSI Perceptive Systems , 1997 .

[24]  E. Culurciello,et al.  A biomorphic digital image sensor , 2003, IEEE J. Solid State Circuits.

[25]  Amine Bermak,et al.  Arbitrated Time-to-First Spike CMOS Image Sensor With On-Chip Histogram Equalization , 2007, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[26]  Jean-Luc Nagel,et al.  An SoC combining a 132dB QVGA pixel array and a 32b DSP/MCU processor for vision applications , 2009, 2009 IEEE International Solid-State Circuits Conference - Digest of Technical Papers.

[27]  Eugenio Culurciello,et al.  An Address-Event Fall Detector for Assisted Living Applications , 2008, IEEE Transactions on Biomedical Circuits and Systems.

[28]  Tobias Delbrück,et al.  Frame-free dynamic digital vision , 2008 .

[29]  Eric R. Fossum,et al.  CMOS image sensors: electronic camera on a chip , 1995, Proceedings of International Electron Devices Meeting.

[30]  Eric Grenet,et al.  High dynamic range vision sensor for automotive applications , 2005, European Workshop on Photonics in the Automobile.

[31]  J. Harris,et al.  A Time-to-First-Spike CMOS Image Sensor , 2007 .

[32]  Martin Litzenberger,et al.  A Dual-Line Optical Transient Sensor with On-Chip Precision Time-Stamp Generation , 2007, 2007 IEEE International Solid-State Circuits Conference. Digest of Technical Papers.

[33]  R. Sarpeshkar,et al.  An analog bionic ear processor with zero-crossing detection , 2005, ISSCC. 2005 IEEE International Digest of Technical Papers. Solid-State Circuits Conference, 2005..

[34]  Tobi Delbrück,et al.  A 128$\times$ 128 120 dB 15 $\mu$s Latency Asynchronous Temporal Contrast Vision Sensor , 2008, IEEE Journal of Solid-State Circuits.

[35]  Bernabé Linares-Barranco,et al.  A Spatial Contrast Retina With On-Chip Calibration for Neuromorphic Spike-Based AER Vision Systems , 2007, IEEE Transactions on Circuits and Systems I: Regular Papers.

[36]  Heinrich Garn,et al.  Estimation of Vehicle Speed Based on Asynchronous Data from a Silicon Retina Optical Sensor , 2006, 2006 IEEE Intelligent Transportation Systems Conference.

[37]  K. Boahen,et al.  A 360-Channel Speech Preprocessor that Emulates the Cochlear Amplifier , 2006, 2006 IEEE International Solid State Circuits Conference - Digest of Technical Papers.

[38]  A.N. Belbachir,et al.  Embedded Vision System for Real-Time Object Tracking using an Asynchronous Transient Vision Sensor , 2006, 2006 IEEE 12th Digital Signal Processing Workshop & 4th IEEE Signal Processing Education Workshop.

[39]  Bernabé Linares-Barranco,et al.  A signed spatial contrast event spike retina chip , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[40]  T. Delbruck,et al.  > Replace This Line with Your Paper Identification Number (double-click Here to Edit) < 1 , 2022 .

[41]  Tobi Delbrück,et al.  An embedded AER dynamic vision sensor for low-latency pole balancing , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[42]  Massimo Gottardi,et al.  A 100μW 64×128-Pixel Contrast-Based Asynchronous Binary Vision Sensor for Wireless Sensor Networks , 2008, 2008 IEEE International Solid-State Circuits Conference - Digest of Technical Papers.

[43]  R. Etienne-Cummings,et al.  Temporal change threshold detection imager , 2005, ISSCC. 2005 IEEE International Digest of Technical Papers. Solid-State Circuits Conference, 2005..

[44]  Mehdi Azadmehr,et al.  A foveated AER imager chip [address event representation] , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[45]  John Wawrzynek,et al.  Silicon Auditory Processors as Computer Peripherals , 1992, NIPS.

[46]  Eugenio Culurciello,et al.  Second generation of high dynamic range, arbitrated digital imager , 2004, 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512).

[47]  Tobi Delbrück,et al.  Fast sensory motor control based on event-based hybrid neuromorphic-procedural system , 2007, 2007 IEEE International Symposium on Circuits and Systems.

[48]  Pierre-Yves Burgi,et al.  A 128 /spl times/ 128 pixel 120 dB dynamic range vision sensor chip for image contrast and orientation extraction , 2003, 2003 IEEE International Solid-State Circuits Conference, 2003. Digest of Technical Papers. ISSCC..

[49]  Daniel Matolin,et al.  True correlated double sampling and comparator design for time-based image sensors , 2009, 2009 IEEE International Symposium on Circuits and Systems.

[50]  John G. Harris,et al.  A time-to-first spike CMOS imager , 2004, 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512).