Stereo Correspondence Evaluation Methods: A Systematic Review

The stereo correspondence problem has received significant attention in literature during approximately three decades. During that period of time, the development on stereo matching algorithms has been quite considerable. In contrast, the proposals on evaluation methods for stereo matching algorithms are not so many. This is not trivial issue, since an objective assessment of algorithms is required not only to measure improvements on the area, but also to properly identify where the gaps really are, and consequently, guiding the research. In this paper, a systematic review on evaluation methods for stereo matching algorithms is presented. The contributions are not only on the found results, but also on how it is explained and presented: aiming to be useful for the researching community on visual computing, in which such systematic review process is not yet broadly adopted.

[1]  Reinhard Klette,et al.  Half-resolution semi-global stereo matching , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[2]  Kai Petersen,et al.  Systematic Mapping Studies in Software Engineering , 2008, EASE.

[3]  Richard Szeliski,et al.  Prediction error as a quality metric for motion and stereo , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[4]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[5]  María P. Trujillo,et al.  A Measure for Accuracy Disparity Maps Evaluation , 2011, CIARP.

[6]  Uwe Franke,et al.  Performance Evaluation of Stereo Algorithms for Automotive Applications , 2009, ICVS.

[7]  Philip Kelly.  Pedestrian detection and tracking using stereo vision techniques , 2008 .

[8]  Barbara Kitchenham,et al.  Procedures for Performing Systematic Reviews , 2004 .

[9]  Luc Van Gool,et al.  Dynamic 3D Scene Analysis from a Moving Vehicle , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Dariu Gavrila,et al.  A new benchmark for stereo-based pedestrian detection , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[11]  Ramin Samadani,et al.  An evaluation of stereo matching methods for view interpolation , 2013, 2013 IEEE International Conference on Image Processing.

[12]  M. Nielsen,et al.  Ground truth evaluation of computer vision based 3D reconstruction of synthesized and real plant images , 2007, Precision Agriculture.

[13]  I. Cabezas,et al.  BMPRE: An Error measure for evaluating disparity maps , 2012, 2012 IEEE 11th International Conference on Signal Processing.

[14]  Zhi-Gang Zheng,et al.  A region based stereo matching algorithm using cooperative optimization , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Richard Szeliski,et al.  High-accuracy stereo depth maps using structured light , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[16]  Chunping Hou,et al.  An Objective Evaluation for Disparity Map Based on the Disparity Gradient and Disparity Acceleration , 2009, 2009 International Conference on Information Technology and Computer Science.

[17]  Anita Sellent,et al.  Quality Assessment of Non-dense Image Correspondences , 2012, ECCV Workshops.

[18]  Daniel Cremers,et al.  Efficient Dense Scene Flow from Sparse or Dense Stereo Data , 2008, ECCV.

[19]  Bok-Suk Shin,et al.  Evaluation of two stereo matchers on long real-world video sequences , 2015, Pattern Recognit..

[20]  Richard Szeliski,et al.  An Experimental Comparison of Stereo Algorithms , 1999, Workshop on Vision Algorithms.

[21]  Bernd Jähne,et al.  Stereo Ground Truth with Error Bars , 2014, ACCV.

[22]  Margrit Gelautz,et al.  Ground Truth Evaluation for Event-Based Silicon Retina Stereo Data , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[23]  Reinhard Klette,et al.  Benchmarking Stereo Data (Not the Matching Algorithms) , 2010, DAGM-Symposium.

[24]  Yee-Hong Yang,et al.  Evaluation of constructable match cost measures for stereo correspondence using cluster ranking , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Pin Xu,et al.  Objective Quality Assessment of Noised Stereoscopic Images , 2011, 2011 Third International Conference on Measuring Technology and Mechatronics Automation.

[26]  Reinhard Klette,et al.  Robustness evaluation of stereo algorithms on long stereo sequences , 2009, 2009 IEEE Intelligent Vehicles Symposium.

[27]  Dariu Gavrila,et al.  Real-time dense stereo for intelligent vehicles , 2006, IEEE Transactions on Intelligent Transportation Systems.

[28]  Toby P. Breckon,et al.  A foreground object based quantitative assessment of dense stereo approaches for use in automotive environments , 2013, 2013 IEEE International Conference on Image Processing.

[29]  Ruigang Yang,et al.  A Performance Study on Different Cost Aggregation Approaches Used in Real-Time Stereo Matching , 2007, International Journal of Computer Vision.

[30]  Rahul Nair,et al.  Ensemble Learning for Confidence Measures in Stereo Vision , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Heiko Hirschmüller,et al.  Evaluation of Stereo Matching Costs on Images with Radiometric Differences , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Xi Wang,et al.  High-Resolution Stereo Datasets with Subpixel-Accurate Ground Truth , 2014, GCPR.

[33]  Georgy L. Gimel'farb,et al.  Generation of an accurate Facialground Truth for Stereo Algorithm Evaluation , 2004, ICCVG.

[34]  Patrick Vandewalle,et al.  Disparity map quality for image-based rendering based on multiple metrics , 2014, 2014 International Conference on 3D Imaging (IC3D).

[35]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Radim Sára,et al.  Feasibility Boundary in Dense and Semi-Dense Stereo Matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  John Morris,et al.  Robustness to noise of stereo matching , 2003, 12th International Conference on Image Analysis and Processing, 2003.Proceedings..

[38]  R. Klette,et al.  Evaluation of stereo confidence measures on synthetic and recorded image data , 2012, 2012 International Conference on Informatics, Electronics & Vision (ICIEV).

[39]  Chris Varekamp,et al.  Detection and correction of disparity estimation errors via supervised learning , 2013, 2013 International Conference on 3D Imaging.

[40]  Xiaoyan Hu,et al.  A Quantitative Evaluation of Confidence Measures for Stereo Vision , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Alan F. Smeaton,et al.  A Framework for Evaluating Stereo-Based Pedestrian Detection Techniques , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[42]  Manuel A. Aguilar,et al.  Generation and Quality Assessment of Stereo-Extracted DSM From GeoEye-1 and WorldView-2 Imagery , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[43]  Ajmal S. Mian,et al.  Face Recognition Using Sparse Approximated Nearest Points between Image Sets , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Reinhard Klette,et al.  A Third Eye for Performance Evaluation in Stereo Sequence Analysis , 2009, CAIP.

[45]  Alan C. Bovik,et al.  Range image quality assessment by Structural Similarity , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[46]  Reinhard Klette,et al.  Ground Truth Evaluation of Stereo Algorithms for Real World Applications , 2010, ACCV Workshops.

[47]  Federico Tombari,et al.  Stereo for robots: Quantitative evaluation of efficient and low-memory dense stereo algorithms , 2010, 2010 11th International Conference on Control Automation Robotics & Vision.

[48]  Stefano Tubaro,et al.  No-reference quality metric for depth maps , 2013, 2013 IEEE International Conference on Image Processing.