Accurate Dense Stereo Matching Based on Image Segmentation Using an Adaptive Multi-Cost Approach

This paper presents a segmentation-based stereo matching algorithm using an adaptive multi-cost approach, which is exploited for obtaining accuracy disparity maps. The main contribution is to integrate the appealing properties of multi-cost approach into the segmentation-based framework. Firstly, the reference image is segmented by using the mean-shift algorithm. Secondly, the initial disparity of each segment is estimated by an adaptive multi-cost method, which consists of a novel multi-cost function and an adaptive support window cost aggregation strategy. The multi-cost function increases the robustness of the initial raw matching costs calculation and the adaptive window reduces the matching ambiguity effectively. Thirdly, an iterative outlier suppression and disparity plane parameters fitting algorithm is designed to estimate the disparity plane parameters. Lastly, an energy function is formulated in segment domain, and the optimal plane label is approximated by belief propagation. The experimental results with the Middlebury stereo datasets, along with synthesized and real-world stereo images, demonstrate the effectiveness of the proposed approach.

[1]  Petros Daras,et al.  Enhanced disparity estimation in stereo images , 2015, Image Vis. Comput..

[2]  Xu Chen,et al.  Hardware Acceleration for Accurate Stereo Vision System using Mini-Census Adaptive Support Region , 2013 .

[3]  Yann LeCun,et al.  Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches , 2015, J. Mach. Learn. Res..

[4]  Richard Szeliski,et al.  High-accuracy stereo depth maps using structured light , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[5]  Christos Stentoumis,et al.  Stereo matching based on census transformation of image gradients , 2015, Optical Metrology.

[6]  Yuichi Taguchi,et al.  Stereo reconstruction with mixed pixels using adaptive over-segmentation , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Christopher Joseph Pal,et al.  Learning Conditional Random Fields for Stereo , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Jing Zhang,et al.  Accurate real-time stereo correspondence using intra- and inter-scanline optimization , 2012, Journal of Zhejiang University SCIENCE C.

[9]  Chang-Su Kim,et al.  Adaptive smoothness constraints for efficient stereo matching using texture and edge information , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[10]  Masatoshi Okutomi,et al.  A Simple Stereo Algorithm to Recover Precise Object Boundaries and Smooth Surfaces , 2004, International Journal of Computer Vision.

[11]  Xi Wang,et al.  High-Resolution Stereo Datasets with Subpixel-Accurate Ground Truth , 2014, GCPR.

[12]  Zhi-Gang Zheng,et al.  A region based stereo matching algorithm using cooperative optimization , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Andrew W. Fitzgibbon,et al.  PMBP: PatchMatch Belief Propagation for Correspondence Field Estimation , 2014, International Journal of Computer Vision.

[14]  Olga Veksler,et al.  Fast variable window for stereo correspondence using integral images , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[15]  Jonathan T. Barron,et al.  The Fast Bilateral Solver , 2015, ECCV.

[16]  Miao Liao,et al.  Real-time stereo using approximated joint bilateral filtering and dynamic programming , 2012, Journal of Real-Time Image Processing.

[17]  Xukun Shen,et al.  PM-PM: PatchMatch With Potts Model for Object Segmentation and Stereo Matching , 2015, IEEE Transactions on Image Processing.

[18]  Heiko Hirschmüller,et al.  Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Daolei Wang,et al.  Obtaining depth map from segment-based stereo matching using graph cuts , 2011, J. Vis. Commun. Image Represent..

[20]  Takeshi Naemura,et al.  Graph Cut Based Continuous Stereo Matching Using Locally Shared Labels , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Hongyang Chao,et al.  MeshStereo: A Global Stereo Model with Mesh Alignment Regularization for View Interpolation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[22]  Andreas Klaus,et al.  Segment-Based Stereo Matching Using Belief Propagation and a Self-Adapting Dissimilarity Measure , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[23]  Li Zhang,et al.  PMSC: PatchMatch-Based Superpixel Cut for Accurate Stereo Matching , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[24]  Heiko Hirschmüller,et al.  Evaluation of Cost Functions for Stereo Matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  In-So Kweon,et al.  Adaptive Support-Weight Approach for Correspondence Search , 2006, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  C. Stentoumis,et al.  On accurate dense stereo-matching using a local adaptive multi-cost approach , 2014 .

[27]  Emile A. Hendriks,et al.  Adaptive stereo similarity fusion using confidence measures , 2015, Comput. Vis. Image Underst..

[28]  Barbara Engelhardt Martin.  Bayesian group factor analysis with structured sparsity , 2016 .

[29]  Neil A. Dodgson,et al.  Real-Time Spatiotemporal Stereo Matching Using the Dual-Cross-Bilateral Grid , 2010, ECCV.

[30]  Kai Gao,et al.  Stereo matching algorithm based on illumination normal similarity and adaptive support weight , 2013 .

[31]  Alexandrina Rogozan,et al.  A robust cost function for stereo matching of road scenes , 2014, Pattern Recognit. Lett..

[32]  Jae Wook Jeon,et al.  Domain Transformation-Based Efficient Cost Aggregation for Local Stereo Matching , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[33]  Joost van de Weijer,et al.  Accurate Stereo Matching by Two-Step Energy Minimization , 2015, IEEE Transactions on Image Processing.

[34]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Carsten Rother,et al.  Fast cost-volume filtering for visual correspondence and beyond , 2011, CVPR 2011.

[36]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[37]  Luuk J. Spreeuwers,et al.  Local Stereo Matching Using Adaptive Local Segmentation , 2012 .

[38]  Gauthier Lafruit,et al.  Cross-Based Local Stereo Matching Using Orthogonal Integral Images , 2009, IEEE Transactions on Circuits and Systems for Video Technology.