Stereo matching using cost volume watershed and region merging

Segment based disparity estimation methods have been proposed in many different ways. Most of these studies are built upon the hypothesis that no large disparity jump exists within a segment. When this hypothesis does not hold, it is difficult for these methods to estimate disparities correctly. Therefore, these methods work well only when the images are initially over segmented but do not work well for under segmented cases. To solve this problem, we present a new segment based stereo matching method which consists of two algorithms: a cost volume watershed algorithm (CVW) and a region merging (RM) algorithm. For incorrectly under segmented regions where pixels on different objects are grouped into one segment, the CVW algorithm regroups the pixels on different objects into different segments and provides disparity estimation to the pixels in different segments accordingly. For unreliable and occluded regions, we merge them into neighboring reliable segments for robust disparity estimation. The comparison between our method and the current state-of-the-art methods shows that our method is very competitive and is robust particularly when the images are initially under segmented. HighlightsA novel segmentation based stereo matching algorithm is proposed.Cost volume watershed improves the robustness to the under segmentation problem.Fitting curved surfaces by using multiple planes.Reliable plane estimation by region merging globally.

[1]  Peter N. Belhumeur,et al.  A Bayesian approach to binocular steropsis , 1996, International Journal of Computer Vision.

[2]  Davi Geiger,et al.  Occlusions, Discontinuities, and Epipolar Lines in Stereo , 1998, ECCV.

[3]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[4]  Li Hong,et al.  Segment-based stereo matching using graph cuts , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[5]  Takeo Kanade,et al.  A Stereo Matching Algorithm with an Adaptive Window: Theory and Experiment , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Tomaso Poggio,et al.  Cooperative computation of stereo disparity , 1988 .

[7]  Margrit Gelautz,et al.  A layered stereo algorithm using image segmentation and global visibility constraints , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[8]  Zheng Zhi.  A Region Based Stereo Matching Algorithm Using Cooperative Optimization , 2009 .

[9]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Andreas Klaus,et al.  Segment-Based Stereo Matching Using Belief Propagation and a Self-Adapting Dissimilarity Measure , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[11]  KanadeT.,et al.  A Stereo Matching Algorithm with an Adaptive Window , 1994 .

[12]  Jian Sun,et al.  Symmetric stereo matching for occlusion handling , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[13]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[14]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Cevahir Çigla,et al.  Information permeability for stereo matching , 2013, Signal Process. Image Commun..

[16]  Aaron F. Bobick,et al.  Large Occlusion Stereo , 1999, International Journal of Computer Vision.

[17]  Pushmeet Kohli,et al.  Surface stereo with soft segmentation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  Richard Szeliski,et al.  A layered approach to stereo reconstruction , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[19]  Emanuele Trucco,et al.  Efficient stereo with multiple windowing , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  Ingemar J. Cox,et al.  A maximum likelihood N-camera stereo algorithm , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Carlo Tomasi,et al.  Surfaces with occlusions from layered stereo , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Rafael Cabeza,et al.  Near Real-Time Stereo Matching Using Geodesic Diffusion , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Hai Tao,et al.  A global matching framework for stereo computation , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[24]  Andreas Geiger,et al.  Efficient Large-Scale Stereo Matching , 2010, ACCV.

[25]  Luc Vincent,et al.  Watersheds in Digital Spaces: An Efficient Algorithm Based on Immersion Simulations , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Heiko Hirschmüller,et al.  Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Pedro F. Felzenszwalb,et al.  Efficient belief propagation for early vision , 2004, CVPR 2004.

[28]  Sing Bing Kang,et al.  Stereo for Image-Based Rendering using Image Over-Segmentation , 2007, International Journal of Computer Vision.

[29]  Sang Uk Lee,et al.  A dense stereo matching using two-pass dynamic programming with generalized ground control points , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[30]  A FischlerMartin,et al.  Random sample consensus , 1981 .

[31]  Changming Sun,et al.  Tree structural watershed for stereo matching , 2012, IVCNZ '12.

[32]  Richard Szeliski,et al.  High-quality video view interpolation using a layered representation , 2004, SIGGRAPH 2004.

[33]  Qingxiong Yang,et al.  Near Real-time Stereo for Weakly-Textured Scenes , 2008, BMVC.

[34]  Nanning Zheng,et al.  Stereo Matching Using Belief Propagation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  M. Okutomi,et al.  A simple stereo algorithm to recover precise object boundaries and smooth surfaces , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[36]  In-So Kweon,et al.  Adaptive Support-Weight Approach for Correspondence Search , 2006, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Gauthier Lafruit,et al.  Cross-Based Local Stereo Matching Using Orthogonal Integral Images , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[38]  Ruigang Yang,et al.  Stereo Matching with Color-Weighted Correlation, Hierarchical Belief Propagation and Occlusion Handling , 2006, CVPR.

[39]  Andrew W. Fitzgibbon,et al.  Global stereo reconstruction under second order smoothness priors , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  SoillePierre,et al.  Watersheds in Digital Spaces , 1991 .

[41]  Ruigang Yang,et al.  Global stereo matching leveraged by sparse ground control points , 2011, CVPR 2011.

[42]  Heiko Hirschmüller,et al.  Evaluation of Cost Functions for Stereo Matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.