A Quantitative Evaluation of Confidence Measures for Stereo Vision

We present an extensive evaluation of 17 confidence measures for stereo matching that compares the most widely used measures as well as several novel techniques proposed here. We begin by categorizing these methods according to which aspects of stereo cost estimation they take into account and then assess their strengths and weaknesses. The evaluation is conducted using a winner-take-all framework on binocular and multibaseline datasets with ground truth. It measures the capability of each confidence method to rank depth estimates according to their likelihood for being correct, to detect occluded pixels, and to generate low-error depth maps by selecting among multiple hypotheses for each pixel. Our work was motivated by the observation that such an evaluation is missing from the rapidly maturing stereo literature and that our findings would be helpful to researchers in binocular and multiview stereo.

[1]  D Marr,et al.  Cooperative computation of stereo disparity. , 1976, Science.

[2]  Qian Chen,et al.  A volumetric stereo matching method: application to image-based modeling , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[3]  Roberto Manduchi,et al.  Distinctiveness maps for image matching , 1999, Proceedings 10th International Conference on Image Analysis and Processing.

[4]  Zhengyou Zhang,et al.  A Progressive Scheme for Stereo Matching , 2000, SMILE.

[5]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[6]  Geoffrey Egnal,et al.  Detecting Binocular Half-Occlusions: Empirical Comparisons of Five Approaches , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Simon Lacroix,et al.  Using multiple disparity hypotheses for improved indoor stereo , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[8]  Richard Szeliski,et al.  High-accuracy stereo depth maps using structured light , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[9]  Darius Burschka,et al.  Advances in Computational Stereo , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Jonathan M. Garibaldi,et al.  Real-Time Correlation-Based Stereo Vision with Reduced Border Errors , 2002, International Journal of Computer Vision.

[11]  Aaron F. Bobick,et al.  Large Occlusion Stereo , 1999, International Journal of Computer Vision.

[12]  Geoffrey Egnal,et al.  A stereo confidence metric using single view imagery with comparison to five alternative approaches , 2004, Image Vis. Comput..

[13]  Larry H. Matthies,et al.  Stereo vision for planetary rovers: Stochastic modeling to near real-time implementation , 1991, Optics & Photonics.

[14]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[15]  Richard Szeliski,et al.  Stereo Matching with Nonlinear Diffusion , 1998, International Journal of Computer Vision.

[16]  Long Quan,et al.  Region-based progressive stereo matching , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[17]  Long Quan,et al.  A quasi-dense approach to surface reconstruction from uncalibrated images , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Minglun Gong,et al.  Fast Unambiguous Stereo Matching Using Reliability-Based Dynamic Programming , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Jian Sun,et al.  Symmetric stereo matching for occlusion handling , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[20]  Christophe Rosenberger,et al.  Detecting Half-Occlusion with a Fast Region-Based Fusion Procedure , 2006, BMVC.

[21]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[22]  J. Weickert,et al.  A Confidence Measure for Variational Optic flow Methods , 2006 .

[23]  Ruigang Yang,et al.  A Performance Study on Different Cost Aggregation Approaches Used in Real-Time Stereo Matching , 2007, International Journal of Computer Vision.

[24]  Jan-Michael Frahm,et al.  Real-Time Visibility-Based Fusion of Depth Maps , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[25]  Radim Sára,et al.  Feasibility Boundary in Dense and Semi-Dense Stereo Matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Radim Sára,et al.  Efficient Sampling of Disparity Space for Fast And Accurate Matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  François Cabestaing,et al.  A Colour Correlation-Based Stereo Matching Using 1D Windows , 2007, 2007 Third International IEEE Conference on Signal-Image Technologies and Internet-Based System.

[28]  Heiko Hirschmüller,et al.  Evaluation of Cost Functions for Stereo Matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Jan-Michael Frahm,et al.  Detailed Real-Time Urban 3D Reconstruction from Video , 2007, International Journal of Computer Vision.

[30]  Michael Goesele,et al.  Multi-View Stereo for Community Photo Collections , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[31]  Jan-Michael Frahm,et al.  Real-Time Plane-Sweeping Stereo with Multiple Sweeping Directions , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Yi Deng,et al.  Stereo Correspondence with Occlusion Handling in a Symmetric Patch-Based Graph-Cuts Model , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  S. Chambon,et al.  EVALUATION OF DIFFERENT METHODS FOR USING COLOUR INFORMATION IN GLOBAL STEREO MATCHING APPROACHES , 2008 .

[34]  In-So Kweon,et al.  Distinctive Similarity Measure for stereo matching under point ambiguity , 2008, Comput. Vis. Image Underst..

[35]  Li Xu,et al.  Stereo Matching: An Outlier Confidence Approach , 2008, ECCV.

[36]  Yee-Hong Yang,et al.  Evaluation of constructable match cost measures for stereo correspondence using cluster ranking , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Pascal Fua,et al.  On benchmarking camera calibration and multi-view stereo for high resolution imagery , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[38]  Federico Tombari,et al.  Classification and evaluation of cost aggregation methods for stereo correspondence , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  Pushmeet Kohli,et al.  Measuring uncertainty in graph cut solutions , 2008, Comput. Vis. Image Underst..

[40]  Tomás Pajdla,et al.  Scalable multi-view stereo , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[41]  Heiko Hirschmüller,et al.  Evaluation of Stereo Matching Costs on Images with Radiometric Differences , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Philippos Mordohai,et al.  The Self-Aware Matching Measure for stereo , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[43]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Xiaoyan Hu,et al.  Evaluation of stereo confidence indoors and outdoors , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[45]  M. Bleyer.  Does Color Really Help in Dense Stereo Matching ? , 2010 .

[46]  Marc Pollefeys,et al.  Segmenting video into classes of algorithm-suitability , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[47]  Tim Weyrich,et al.  Capturing Time-of-Flight data with confidence , 2011, CVPR 2011.