Methodologies for Evaluating Disparity Estimation Algorithms

The use of disparity estimation algorithms is required in the 3D recovery process from stereo images. These algorithms tackle the correspondence problem by computing a disparity map. The accuracy assessment of a disparity estimation process has multiple applications such as comparing among different algorithms’ performance, tuning algorithm’s parameters within a particular context, and determining the impact of components, among others. Disparity estimation algorithms can be assessed by following an evaluation methodology. This chapter is dedicated to present and discuss methodologies for evaluating disparity estimation algorithms. The discussion begins with a review of the state-of-the-art. The constitutive components of a methodology are analysed. Finally, advantages and drawbacks of existing methodologies are presented. DOI: 10.4018/978-1-4666-2672-0.ch010

[1]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[2]  Richard Szeliski,et al.  Prediction error as a quality metric for motion and stereo , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[3]  Federico Tombari,et al.  Classification and evaluation of cost aggregation methods for stereo correspondence , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Reinhard Klette,et al.  A Third Eye for Performance Evaluation in Stereo Sequence Analysis , 2009, CAIP.

[5]  Alan C. Bovik,et al.  Range image quality assessment by Structural Similarity , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[6]  Patrick Le Callet,et al.  Using disparity for quality assessment of stereoscopic images , 2008, 2008 15th IEEE International Conference on Image Processing.

[7]  Pascal Fua,et al.  Measuring the Self-Consistency of Stereo Algorithms , 2000, ECCV.

[8]  Federico Tombari,et al.  A 3D reconstruction system based on improved spacetime stereo , 2010, 2010 11th International Conference on Control Automation Robotics & Vision.

[9]  Reinhard Klette,et al.  Ground Truth Evaluation of Stereo Algorithms for Real World Applications , 2010, ACCV Workshops.

[10]  Federico Tombari,et al.  Stereo for robots: Quantitative evaluation of efficient and low-memory dense stereo algorithms , 2010, 2010 11th International Conference on Control Automation Robotics & Vision.

[11]  Reinhard Klette,et al.  Benchmarking Stereo Data (Not the Matching Algorithms) , 2010, DAGM-Symposium.

[12]  Richard Szeliski,et al.  Stereo Matching with Transparency and Matting , 1999, International Journal of Computer Vision.

[13]  Richard Szeliski,et al.  An Experimental Comparison of Stereo Algorithms , 1999, Workshop on Vision Algorithms.

[14]  María P. Trujillo,et al.  A Non-linear Quantitative Evaluation Approach for Disparity Estimation - Pareto Dominance Applied in Stereo Vision , 2011, VISAPP.

[15]  María P. Trujillo,et al.  A Measure for Accuracy Disparity Maps Evaluation , 2011, CIARP.

[16]  Yinghua Shen,et al.  Subjective assessment of noised stereo images , 2011, 2011 International Conference on Multimedia Technology.

[17]  Yee-Hong Yang,et al.  Evaluation of constructable match cost measures for stereo correspondence using cluster ranking , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Heiko Hirschmüller,et al.  Evaluation of Stereo Matching Costs on Images with Radiometric Differences , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Vladimir Kolmogorov,et al.  Computing visual correspondence with occlusions using graph cuts , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[20]  Nicolas Papadakis,et al.  Multi-label Depth Estimation for Graph Cuts Stereo Problems , 2010, Journal of Mathematical Imaging and Vision.

[21]  Richard Szeliski,et al.  High-accuracy stereo depth maps using structured light , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[22]  Chunping Hou,et al.  An Objective Evaluation for Disparity Map Based on the Disparity Gradient and Disparity Acceleration , 2009, 2009 International Conference on Information Technology and Computer Science.

[23]  A. Verri,et al.  A compact algorithm for rectification of stereo pairs , 2000 .

[24]  Alan C. Bovik,et al.  Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures , 2009, IEEE Signal Processing Magazine.

[25]  Carsten Rother,et al.  PatchMatch Stereo - Stereo Matching with Slanted Support Windows , 2011, BMVC.

[26]  Alan F. Smeaton,et al.  A Framework for Evaluating Stereo-Based Pedestrian Detection Techniques , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[27]  Margaret Florian,et al.  An Evaluation Methodology for Stereo Correspondence Algorithms , 2012, VISAPP.

[28]  Dariu Gavrila,et al.  Real-time dense stereo for intelligent vehicles , 2006, IEEE Transactions on Intelligent Transportation Systems.

[29]  Abdul Rehman,et al.  Reduced-reference SSIM estimation , 2010, 2010 IEEE International Conference on Image Processing.

[30]  In-So Kweon,et al.  Stereo Matching with the Distinctive Similarity Measure , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[31]  Pascal Fua,et al.  On benchmarking camera calibration and multi-view stereo for high resolution imagery , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Margrit Gelautz,et al.  A layered stereo algorithm using image segmentation and global visibility constraints , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[33]  Oscar E. Ruiz,et al.  Statistical Tuning of Adaptive-Weight Depth Map Algorithm , 2011, CAIP.