A census-based stereo vision algorithm using modified Semi-Global Matching and plane fitting to improve matching quality

This paper introduces a new segmentation-based approach for disparity optimization in stereo vision. The main contribution is a significant enhancement of the matching quality at occlusions and textureless areas by segmenting either the left color image or the calculated texture image. The local cost calculation is done with a Census-based correlation method and is compared with standard sum of absolute differences. The confidence of a match is measured and only non-confident or non-textured pixels are estimated by calculating a disparity plane for the corresponding segment. The quality of the local optimized matches is increased by a modified Semi-Global Matching (SGM) step with subpixel accuracy. In contrast to standard SGM, not the whole image is used for disparity optimization but horizontal stripes of the image. It is shown that this modification significantly reduces the memory consumption by nearly constant matching quality and thus enables embedded realization. Using the Middlebury ranking as evaluation criterion, it is shown that the proposed algorithm performs well in comparison to the pure Census correlation. It reaches a top ten rank if subpixel accuracy is supposed. Furthermore, the matching quality of the algorithm, especially of the texture-based plane fitting, is shown on two real-world scenes where a significant enhancement could be achieved.

[1]  Ramin Zabih,et al.  Non-parametric Local Transforms for Computing Visual Correspondence , 1994, ECCV.

[2]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[3]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Heiko Hirschmüller,et al.  Evaluation of Cost Functions for Stereo Matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  William T. Freeman,et al.  Understanding belief propagation and its generalizations , 2003 .

[6]  Darius Burschka,et al.  Advances in Computational Stereo , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Qingxiong Yang,et al.  Near Real-time Stereo for Weakly-Textured Scenes , 2008, BMVC.

[8]  Ines Ernst,et al.  Mutual Information Based Semi-Global Stereo Matching on the GPU , 2008, ISVC.

[9]  Alfred Schmitt,et al.  Real-Time Stereo by using Dynamic Programming , 2003, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[10]  Heiko Hirschmüller,et al.  Stereo Vision in Structured Environments by Consistent Semi-Global Matching , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[11]  Andreas Klaus,et al.  Segment-Based Stereo Matching Using Belief Propagation and a Self-Adapting Dissimilarity Measure , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[12]  Daniel P. Huttenlocher,et al.  Efficient Belief Propagation for Early Vision , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[13]  Margrit Gelautz,et al.  A layered stereo matching algorithm using image segmentation and global visibility constraints , 2005 .

[14]  Heiko Hirschmüller,et al.  Evaluation of Stereo Matching Costs on Images with Radiometric Differences , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Vladimir Kolmogorov,et al.  Computing visual correspondence with occlusions using graph cuts , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[16]  Carlo Tomasi,et al.  Depth Discontinuities by Pixel-to-Pixel Stereo , 1999, International Journal of Computer Vision.

[17]  Kristian Ambrosch,et al.  An Optimized Software-Based Implementation of a Census-Based Stereo Matching Algorithm , 2008, ISVC.

[18]  Miao Liao,et al.  Real-time Global Stereo Matching Using Hierarchical Belief Propagation , 2006, BMVC.

[19]  W. Kubinger,et al.  Performance evaluation of a census-based stereo matching algorithm on embedded and multi-core hardware , 2009, 2009 Proceedings of 6th International Symposium on Image and Signal Processing and Analysis.