An Efficient Stereo Matching Algorithm Based on Four-Moded Census Transform for High-Resolution Images

By establishing slanted surfaces, PatchMatch stereo (PMS) algorithm can achieve impressive disparity details and high sub-pixel precision. However, it is too unwieldy for practical calculations in handling images with high resolution. In this paper, we improved the PMS algorithm to efficiently handle the high-resolution images. Firstly, four-mode census transform, which can improve matching accuracy and solve the problem of the center pixel distortion effectively, is applied to measure the dissimilarity between pixels, instead of the absolute differences of the gray-value and the gray-value gradient. Utilizing this transform can halve the time of dissimilarity measurement compared to that of PMS. Then, the proposed algorithm adopt the integer disparity plane approximation strategy during the PatchMatch inference procedure. This strategy is applied in the random initialization step, the computation of the matching cost and the process of searching and renewing the minimum matching cost. Finally, the outlier pixels are refined with the post-process steps. Experimental results show that the proposed algorithm is more efficient than the PMS algorithm and generates comparable disparity maps in handling the high-resolution images.Graphical Abstract

[1]  Dong Xu,et al.  A Depth Map Generation Algorithm Based on Saliency Detection for 2D to 3D Conversion , 2017 .

[2]  Cheng Zhang,et al.  Accurate Image-Guided Stereo Matching With Efficient Matching Cost and Disparity Refinement , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  Xin Yu,et al.  3D cost aggregation with multiple minimum spanning trees for stereo matching. , 2017, Applied optics.

[4]  Carsten Rother,et al.  Fast Cost-Volume Filtering for Visual Correspondence and Beyond , 2013, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Yann LeCun,et al.  Computing the stereo matching cost with a convolutional neural network , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Alex Kendall,et al.  End-to-End Learning of Geometry and Context for Deep Stereo Regression , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[7]  Ramin Zabih,et al.  Non-parametric Local Transforms for Computing Visual Correspondence , 1994, ECCV.

[8]  Xi Jin,et al.  A real-time global stereo-matching on FPGA , 2016, Microprocess. Microsystems.

[9]  Madaín Pérez Patricio,et al.  An FPGA stereo matching unit based on fuzzy logic , 2016, Microprocess. Microsystems.

[10]  Haidi Ibrahim,et al.  Stereo matching algorithm based on per pixel difference adjustment, iterative guided filter and graph segmentation , 2017, J. Vis. Commun. Image Represent..

[11]  Carsten Rother,et al.  PatchMatch Stereo - Stereo Matching with Slanted Support Windows , 2011, BMVC.

[12]  Giancarlo Raiconi,et al.  Real-Time Low-Power FPGA Architecture for Stereo Vision , 2017, IEEE Transactions on Circuits and Systems II: Express Briefs.

[13]  Jian Sun,et al.  Symmetric stereo matching for occlusion handling , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[14]  In-So Kweon,et al.  Adaptive Support-Weight Approach for Correspondence Search , 2006, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Xiao Lu,et al.  Robust stereo matching with trinary cross color census and triple image-based refinements , 2017, EURASIP J. Adv. Signal Process..

[16]  Abiel Aguilar-González,et al.  An FPGA Stereo Matching Processor Based on the Sum of Hamming Distances , 2016, ARC.

[17]  Manu Bansal,et al.  A Wavelet-Based Multiresolution Approach to Solve the Stereo Correspondence Problem Using Mutual Information , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[18]  Yingnan Geng.  Local Stereo Matching Based on Information Entropy of Image , 2016 .

[19]  Chaoguang Men,et al.  A Stereo Matching Algorithm Based on Four-Moded Census and Relative Confidence Plane Fitting , 2015 .

[20]  Hengzhu Liu,et al.  Improving stereo matching by incorporating geometry prior into ConvNet , 2017 .

[21]  Raquel Urtasun,et al.  Efficient Deep Learning for Stereo Matching , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Gauthier Lafruit,et al.  Cross-Based Local Stereo Matching Using Orthogonal Integral Images , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[23]  Andreas Zell,et al.  LS-ELAS: Line segment based efficient large scale stereo matching , 2017, ICRA 2017.

[24]  Heiko Hirschmüller,et al.  Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Vladimir Kolmogorov,et al.  Computing visual correspondence with occlusions using graph cuts , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[26]  Yong Seok Heo.  Two-step mutual information-based stereo matching , 2016 .

[27]  Cheng Lei,et al.  Region-Tree Based Stereo Using Dynamic Programming Optimization , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[28]  Nanning Zheng,et al.  Stereo Matching Using Belief Propagation , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Andreas Geiger,et al.  Efficient Large-Scale Stereo Matching , 2010, ACCV.

[30]  Lior Wolf,et al.  Improved Stereo Matching with Constant Highway Networks and Reflective Confidence Learning , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).