Asynchronous frameless event-based optical flow

This paper introduces a process to compute optical flow using an asynchronous event-based retina at high speed and low computational load. A new generation of artificial vision sensors has now started to rely on biologically inspired designs for light acquisition. Biological retinas, and their artificial counterparts, are totally asynchronous and data driven and rely on a paradigm of light acquisition radically different from most of the currently used frame-grabber technologies. This paper introduces a framework for processing visual data using asynchronous event-based acquisition, providing a method for the evaluation of optical flow. The paper shows that current limitations of optical flow computation can be overcome by using event-based visual acquisition, where high data sparseness and high temporal resolution permit the computation of optical flow with micro-second accuracy and at very low computational cost.

[1]  K. Boahen.  Neuromorphic Microchips. , 2005, Scientific American.

[2]  T. Delbruck,et al.  > Replace This Line with Your Paper Identification Number (double-click Here to Edit) < 1 , 2022 .

[3]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[4]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[5]  Tobi Delbrück,et al.  A 128$\times$ 128 120 dB 15 $\mu$s Latency Asynchronous Temporal Contrast Vision Sensor , 2008, IEEE Journal of Solid-State Circuits.

[6]  Manuele Bicego,et al.  Unsupervised scene analysis: a hidden Markov model approach , 2006 .

[7]  Luc Van Gool,et al.  Optical flow based super-resolution: A probabilistic approach , 2007, Comput. Vis. Image Underst..

[8]  Tobi Delbrück,et al.  CAVIAR: A 45k Neuron, 5M Synapse, 12G Connects/s AER Hardware Sensory–Processing– Learning–Actuating System for High-Speed Visual Object Recognition and Tracking , 2009, IEEE Transactions on Neural Networks.

[9]  David J. Fleet,et al.  Performance of optical flow techniques , 1994, International Journal of Computer Vision.

[10]  Bernabé Linares-Barranco,et al.  A Neuromorphic Cortical-Layer Microchip for Spike-Based Event Processing Vision Systems , 2006, IEEE Transactions on Circuits and Systems I: Regular Papers.

[11]  Alexander Shekhovtsov,et al.  A discrete search method for multi-modal non-rigid image registration , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[12]  Botond Roska,et al.  Parallel processing in retinal ganglion cells: how integration of space-time patterns of excitation and inhibition form the spiking output. , 2006, Journal of neurophysiology.

[13]  C. Mead,et al.  Neuromorphic Robot Vision with Mixed Analog- Digital Architecture , 2005 .

[14]  Nahum Kiryati,et al.  Coarse to over-fine optical flow estimation , 2007, Pattern Recognit..

[15]  Rama Chellappa,et al.  Accuracy vs Efficiency Trade-offs in Optical Flow Algorithms , 1996, Comput. Vis. Image Underst..

[16]  Daniel Matolin,et al.  A QVGA 143 dB Dynamic Range Frame-Free PWM Image Sensor With Lossless Pixel-Level Video Compression and Time-Domain CDS , 2011, IEEE Journal of Solid-State Circuits.

[17]  Mohammed Ismail,et al.  Analog VLSI Implementation of Neural Systems , 2011, The Kluwer International Series in Engineering and Computer Science.

[18]  Patrick Pérez,et al.  Joint Estimation-Segmentation of Optic Flow , 1998, ECCV.

[19]  Bertram E. Shi,et al.  Neuromorphic implementation of orientation hypercolumns , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[20]  Bernabé Linares-Barranco,et al.  A 3.6 $\mu$ s Latency Asynchronous Frame-Free Event-Driven Dynamic-Vision-Sensor , 2011, IEEE Journal of Solid-State Circuits.

[21]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[22]  Michael J. Black,et al.  The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields , 1996, Comput. Vis. Image Underst..

[23]  Daniel Matolin,et al.  High-DR frame-free PWM imaging with asynchronous AER intensity encoding and focal-plane temporal redundancy suppression , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[24]  Tobi Delbrück,et al.  Modeling orientation selectivity using a neuromorphic multi-chip system , 2006, 2006 IEEE International Symposium on Circuits and Systems.

[25]  Carver Mead,et al.  Adaptive Retina , 1989, Analog VLSI Implementation of Neural Systems.