A 1D approach to correlation-based stereo matching

In stereovision, indices allowing pixels of the left and right images to be matched are basically one-dimensional features of the epipolar lines. In some situations, these features are not significant or cannot be extracted from the single epipolar line. Therefore, many techniques use 2D neighbourhoods to increase the available information. In this paper, we discuss the systematic use of 2D neighbourhoods for stereo matching. We propose an alternative approach to stereo matching using multiple 1D correlation windows, which yields a semi-dense disparity map and an associated confidence map. A particular technique derived from this approach - using fuzzy filtering and a basic decision rule - is compared to about 80 other methods on the Middlebury image datasets [1]. Results are first presented in the framework of the Middlebury website, then on the Receiver Operating Characteristics (ROC) evaluation [2] and, finally, on stereo image pairs of slanted surfaces. We show that a 1D correlation window is sufficient to provide correct matchings in most cases.

[1]  Radim Sára,et al.  Feasibility Boundary in Dense and Semi-Dense Stereo Matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Sang Uk Lee,et al.  A dense stereo matching using two-pass dynamic programming with generalized ground control points , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[3]  C. V. Jawahar,et al.  Generalised correlation for multi-feature correspondence , 2002, Pattern Recognit..

[4]  Olga Veksler,et al.  Fast variable window for stereo correspondence using integral images , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[5]  Liang-Gee Chen,et al.  Hardware-Efficient Belief Propagation , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[6]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[7]  William T. Freeman,et al.  Comparison of graph cuts with belief propagation for stereo, using identical MRF parameters , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[8]  Tianli Yu,et al.  Efficient Message Representations for Belief Propagation , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[9]  David J. Kriegman,et al.  Introduction of New Editor-in-Chief , 2009, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Heiko Hirschmüller,et al.  Improvements in real-time correlation-based stereo vision , 2001, CVPR 2001.

[11]  Ruigang Yang,et al.  Spatial-Depth Super Resolution for Range Images , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  D. Nistér,et al.  Stereo Matching with Color-Weighted Correlation, Hierarchical Belief Propagation, and Occlusion Handling , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Federico Tombari,et al.  Classification and evaluation of cost aggregation methods for stereo correspondence , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Neill W Campbell,et al.  IEEE International Conference on Computer Vision and Pattern Recognition , 2008 .

[15]  Geoffrey Egnal,et al.  A stereo confidence metric using single view imagery with comparison to five alternative approaches , 2004, Image Vis. Comput..

[16]  Margrit Gelautz,et al.  Local stereo matching using geodesic support weights , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[17]  Emanuele Trucco,et al.  Symmetric Stereo with Multiple Windowing , 2000, Int. J. Pattern Recognit. Artif. Intell..

[18]  Narendra Ahuja,et al.  A constant-space belief propagation algorithm for stereo matching , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[19]  Pascal Fua,et al.  A parallel stereo algorithm that produces dense depth maps and preserves image features , 1993, Machine Vision and Applications.

[20]  In-So Kweon,et al.  Adaptive Support-Weight Approach for Correspondence Search , 2006, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Olivier Colot,et al.  A similarity-based adaptive neighborhood method for correlation-based stereo matching , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[22]  Federico Tombari,et al.  Segmentation-Based Adaptive Support for Accurate Stereo Correspondence , 2007, PSIVT.

[23]  Takeo Kanade,et al.  A Stereo Matching Algorithm with an Adaptive Window: Theory and Experiment , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Geoffrey E. Hinton,et al.  Learning Generative Texture Models with extended Fields-of-Experts , 2009, BMVC.

[25]  Masatoshi Okutomi,et al.  A Simple Stereo Algorithm to Recover Precise Object Boundaries and Smooth Surfaces , 2004, International Journal of Computer Vision.

[26]  Miao Liao,et al.  High-Quality Real-Time Stereo Using Adaptive Cost Aggregation and Dynamic Programming , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[27]  Philippe Bekaert,et al.  Local Stereo Matching with Segmentation-based Outlier Rejection , 2006, The 3rd Canadian Conference on Computer and Robot Vision (CRV'06).

[28]  Vladimir Kolmogorov,et al.  Computing visual correspondence with occlusions using graph cuts , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[29]  Y. Aloimonos,et al.  Stereo correspondence with slanted surfaces: critical implications of horizontal slant , 2004, CVPR 2004.

[30]  Heiko Hirschmüller,et al.  Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Miao Liao,et al.  Real-time Global Stereo Matching Using Hierarchical Belief Propagation , 2006, BMVC.

[32]  Ingemar J. Cox,et al.  A Maximum Likelihood Stereo Algorithm , 1996, Comput. Vis. Image Underst..

[33]  Radim Sára,et al.  Finding the Largest Unambiguous Component of Stereo Matching , 2002, ECCV.

[34]  Sébastien Lefebvre.  Approche monodimensionnelle de la mise en correspondance stéréoscopique par corrélation : application à la détection d'obstacles routiers , 2007 .

[35]  Changming Sun,et al.  Fast Stereo Matching Using Rectangular Subregioning and 3D Maximum-Surface Techniques , 2002, International Journal of Computer Vision.

[36]  Li Li,et al.  Local Stereo Matching with Edge-Based Cost Aggregation and Occlusion Handling , 2009, 2009 2nd International Congress on Image and Signal Processing.

[37]  Katsushi Ikeuchi,et al.  Interactive Shadow Removal from a Single Image Using Hierarchical Graph Cut , 2009, ACCV.

[38]  Rafael Mayoral,et al.  Evaluation of correspondence errors for stereo , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[39]  Radim Sára,et al.  Stratified Dense Matching for Stereopsis in Complex Scenes , 2003, BMVC.

[40]  Alfred Schmitt,et al.  Real-Time Stereo by using Dynamic Programming , 2003, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[41]  Emanuele Trucco,et al.  Rectification with unconstrained stereo geometry , 1997, BMVC.

[42]  Reinhard Männer,et al.  Calculating Dense Disparity Maps from Color Stereo Images, an Efficient Implementation , 2004, International Journal of Computer Vision.