Event-based stereo matching using semiglobal matching

In this article, we focus on the problem of depth estimation from a stereo pair of event-based sensors. These sensors asynchronously capture pixel-level brightness changes information (events) instead of standard intensity images at a specified frame rate. So, these sensors provide sparse data at low latency and high temporal resolution over a wide intrascene dynamic range. However, new asynchronous, event-based processing algorithms are required to process the event streams. We propose a fully event-based stereo three-dimensional depth estimation algorithm inspired by semiglobal matching. Our algorithm considers the smoothness constraints between the nearby events to remove the ambiguous and wrong matches when only using the properties of a single event or local features. Experimental validation and comparison with several state-of-the-art, event-based stereo matching methods are provided on five different scenes of event-based stereo data sets. The results show that our method can operate well in an event-driven way and has higher estimation accuracy.

[1]  Ryad Benosman,et al.  A spiking neural network model of 3D perception for event-based neuromorphic stereo vision systems , 2017, Scientific Reports.

[2]  Tobi Delbruck,et al.  A 240 × 180 130 dB 3 µs Latency Global Shutter Spatiotemporal Vision Sensor , 2014, IEEE Journal of Solid-State Circuits.

[3]  Misha Anne Mahowald,et al.  VLSI analogs of neuronal visual processing: a synthesis of form and function , 1992 .

[4]  Tobi Delbrück,et al.  Cooperative Stereo Matching Using Static and Dynamic Image Features , 1989, Analog VLSI Implementation of Neural Systems.

[5]  Bernabe Linares-Barranco,et al.  On the use of orientation filters for 3D reconstruction in event-driven stereo vision , 2014, Front. Neurosci..

[6]  Jörg Conradt,et al.  Asynchronous Event-based Cooperative Stereo Matching Using Neuromorphic Silicon Retinas , 2016, Neural Processing Letters.

[7]  Daniel Cremers,et al.  Nonlinear Shape Statistics in Mumford-Shah Based Segmentation , 2002, ECCV.

[8]  Andreas Geiger,et al.  Object scene flow for autonomous vehicles , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  C. Mead,et al.  Neuromorphic Robot Vision with Mixed Analog- Digital Architecture , 2005 .

[10]  Garrick Orchard,et al.  Evaluating noise filtering for event-based asynchronous change detection image sensors , 2016, 2016 6th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[11]  Daniel Cremers,et al.  Event-based 3D SLAM with a depth-augmented dynamic vision sensor , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[12]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[13]  Daniel Matolin,et al.  A QVGA 143 dB Dynamic Range Frame-Free PWM Image Sensor With Lossless Pixel-Level Video Compression and Time-Domain CDS , 2011, IEEE Journal of Solid-State Circuits.

[14]  Shengyong Chen,et al.  Event-Based Stereo Depth Estimation Using Belief Propagation , 2017, Front. Neurosci..

[15]  Tobi Delbrück,et al.  A 128$\times$ 128 120 dB 15 $\mu$s Latency Asynchronous Temporal Contrast Vision Sensor , 2008, IEEE Journal of Solid-State Circuits.

[16]  R. Weale.  Vision. A Computational Investigation Into the Human Representation and Processing of Visual Information. David Marr , 1983 .

[17]  Christoph Sulzbachner,et al.  Event-Based Stereo Matching Approaches for Frameless Address Event Stereo Data , 2011, ISVC.

[18]  Margrit Gelautz,et al.  Ground Truth Evaluation for Event-Based Silicon Retina Stereo Data , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[19]  Pedro F. Felzenszwalb,et al.  Efficient belief propagation for early vision , 2004, CVPR 2004.

[20]  Reinhard Klette,et al.  Iterative Semi-Global Matching for Robust Driver Assistance Systems , 2012, ACCV.

[21]  Johannes Stallkamp,et al.  Real-time stereo vision: Optimizing Semi-Global Matching , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[22]  Raúl Rojas,et al.  Large scale Semi-Global Matching on the CPU , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[23]  Garrick Orchard,et al.  HOTS: A Hierarchy of Event-Based Time-Surfaces for Pattern Recognition , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Tobi Delbrück,et al.  Asynchronous Event-Based Binocular Stereo Matching , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Florian Eibensteiner,et al.  Event-driven stereo vision algorithm based on silicon retina sensors , 2017, 2017 27th International Conference Radioelektronika (RADIOELEKTRONIKA).

[26]  Heiko Hirschmüller,et al.  Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Jongkil Park,et al.  Improved contrast sensitivity DVS and its application to event-driven stereo vision , 2013, 2013 IEEE International Symposium on Circuits and Systems (ISCAS2013).