Efficient binocular stereo correspondence matching with 1-D Max-Trees

Abstract Extraction of depth from images is of great importance for various computer vision applications. Methods based on convolutional neural networks are very accurate but have high computation requirements, which can be achieved with GPUs. However, GPUs are difficult to use on devices with low power requirements like robots and embedded systems. In this light, we propose a stereo matching method appropriate for applications in which limited computational and energy resources are available. The algorithm is based on a hierarchical representation of image pairs which is used to restrict disparity search range. We propose a cost function that takes into account region contextual information and a cost aggregation method that preserves disparity borders. We tested the proposed method on the Middlebury and KITTI benchmark data sets and on the TrimBot2020 synthetic data. We achieved accuracy and time efficiency results that show that the method is suitable to be deployed on embedded and robotics systems.

[1]  Narendra Ahuja,et al.  Region-Based Hierarchical Image Matching , 2008, International Journal of Computer Vision.

[2]  Gauthier Lafruit,et al.  Cross-Based Local Stereo Matching Using Orthogonal Integral Images , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  Liang Wang,et al.  A Deep Visual Correspondence Embedding Model for Stereo Matching Costs , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[4]  Jonathan T. Barron,et al.  Depth from motion for smartphone AR , 2018, ACM Trans. Graph..

[5]  Yann LeCun,et al.  Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches , 2015, J. Mach. Learn. Res..

[6]  Liming Chen,et al.  An improved Non-Local Cost Aggregation method for stereo matching based on color and boundary cue , 2013, 2013 IEEE International Conference on Multimedia and Expo (ICME).

[7]  Xi Wang,et al.  High-Resolution Stereo Datasets with Subpixel-Accurate Ground Truth , 2014, GCPR.

[8]  Raquel Urtasun,et al.  Efficient Deep Learning for Stereo Matching , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Jiamao Li,et al.  Efficient Stereo Matching Leveraging Deep Local and Context Information , 2017, IEEE Access.

[10]  Changming Sun A Fast Stereo Matching Method , 1997 .

[11]  Andreas Geiger,et al.  Efficient Large-Scale Stereo Matching , 2010, ACCV.

[12]  Thomas Brox,et al.  A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Andreas Zell,et al.  LS-ELAS: Line segment based efficient large scale stereo matching , 2017, ICRA 2017.

[14]  Heiko Hirschmüller,et al.  Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Philippe Salembier,et al.  Antiextensive connected operators for image and sequence processing , 1998, IEEE Trans. Image Process..

[16]  Laurent Vinet,et al.  Hierarchical region based stereo matching , 1989, Proceedings CVPR '89: IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Jörg Stückler,et al.  Large-scale direct SLAM with stereo cameras , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[18]  Hongtao Lu,et al.  Fast Non-local Stereo Matching based on Hierarchical Disparity Prediction , 2015, ArXiv.

[19]  Marc Pollefeys,et al.  Reactive avoidance using embedded stereo vision for MAV flight , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[20]  Kyoung Mu Lee,et al.  Look Wider to Match Image Patches With Convolutional Neural Networks , 2017, IEEE Signal Processing Letters.

[21]  Michael H. F. Wilkinson A fast component-tree algorithm for high dynamic-range images and second generation connectivity , 2011, 2011 18th IEEE International Conference on Image Processing.

[22]  Robert B. Fisher,et al.  The Second Workshop on 3D Reconstruction Meets Semantics: Challenge Results Discussion , 2018, ECCV Workshops.

[23]  Anders Grunnet-Jepsen,et al.  Intel RealSense Stereoscopic Depth Cameras , 2017, CVPR 2017.

[24]  M.H.F. Wilkinson,et al.  Connected operators , 2009, IEEE Signal Processing Magazine.

[25]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[26]  Sebastian Ramos,et al.  Vision-Based Offline-Online Perception Paradigm for Autonomous Driving , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[27]  Julian Eggert,et al.  A Two-Stage Correlation Method for Stereoscopic Depth Estimation , 2010, 2010 International Conference on Digital Image Computing: Techniques and Applications.

[28]  In-So Kweon,et al.  Adaptive Support-Weight Approach for Correspondence Search , 2006, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Narendra Ahuja,et al.  Two-view Matching , 1988, [1988 Proceedings] Second International Conference on Computer Vision.

[30]  Ali Shahrokni,et al.  Urban 3D semantic modelling using stereo vision , 2013, 2013 IEEE International Conference on Robotics and Automation.

[31]  Robert B. Fisher,et al.  TrimBot2020: an outdoor robot for automatic gardening , 2018, ArXiv.

[32]  Ramakant Nevatia,et al.  Segment-based stereo matching , 1985, Comput. Vis. Graph. Image Process..

[33]  Christian Heipke,et al.  Joint 3d Estimation of Vehicles and Scene Flow , 2015 .

[34]  Alistair Sutherland,et al.  Disparity Estimation by Simultaneous Edge Drawing , 2016, ACCV Workshops.