Quaternary Census Transform Based on the Human Visual System for Stereo Matching

The census transform is a non-parametric local transform that is widely used in stereo matching. This transform encodes the structural information of a local patch into a binary code stream representing the relative intensity ordering of the pixels within the patch. Despite its high performance in stereo matching, the census transform often generates identical binary code streams for two different patches because it simply thresholds the pixels within the patch at the center pixel intensity. To overcome this problem, we introduce a quaternary census transform that encodes the local structural information into a quaternary code stream by employing both the relative intensity ordering and the minimum visibility threshold of the human eye known as the just-noticeable difference. Moreover, because the human eye activates different areas of the retina based on brightness, the patch size for the proposed quaternary census transform adaptively varies depending on the luminance of each pixel. Experimental results on well-known Middlebury stereo datasets prove that the proposed transform outperforms the other census transform-based methods in terms of the accuracy of stereo matching.

[1]  John L. Barbur,et al.  Photopic, Mesopic, and Scotopic Vision and Changes in Visual Performance , 2010 .

[2]  Malur K. Sundareshan,et al.  Adaptive image contrast enhancement based on human visual properties , 1994, IEEE Trans. Medical Imaging.

[3]  Moncef Gabbouj,et al.  Noise-Robust Texture Description Using Local Contrast Patterns via Global Measures , 2014, IEEE Signal Processing Letters.

[4]  Yangzhou Gan,et al.  Segment-Based Disparity Refinement With Occlusion Handling for Stereo Matching , 2019, IEEE Transactions on Image Processing.

[5]  Chang-Su Kim,et al.  Consistent Stereo Matching Under Varying Radiometric Conditions , 2013, IEEE Transactions on Multimedia.

[6]  Christopher Joseph Pal,et al.  Learning Conditional Random Fields for Stereo , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Chaoguang Men,et al.  A Stereo Matching Algorithm Based on Four-Moded Census and Relative Confidence Plane Fitting , 2015 .

[8]  Petros Daras,et al.  Content-Based Guided Image Filtering, Weighted Semi-Global Optimization, and Efficient Disparity Refinement for Fast and Accurate Disparity Estimation , 2016, IEEE Transactions on Multimedia.

[9]  Tianqi Zhang,et al.  Grayscale-Inversion and Rotation Invariant Texture Description Using Sorted Local Gradient Pattern , 2018, IEEE Signal Processing Letters.

[10]  R. C. Murry,et al.  Christensen's physics of diagnostic radiology , 1990 .

[11]  Xiaogang Wang,et al.  Group-Wise Correlation Stereo Network , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Zhenhua Guo,et al.  A Completed Modeling of Local Binary Pattern Operator for Texture Classification , 2010, IEEE Transactions on Image Processing.

[13]  Sankar K. Pal,et al.  Thresholding for edge detection using human psychovisual phenomena , 1986, Pattern Recognit. Lett..

[14]  Rolf G. Kuehni,et al.  The Measurement of Sensation , 2005 .

[15]  Heiko Hirschmüller,et al.  Evaluation of Stereo Matching Costs on Images with Radiometric Differences , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Xi Wang,et al.  High-Resolution Stereo Datasets with Subpixel-Accurate Ground Truth , 2014, GCPR.

[17]  Joachim Weickert,et al.  Why Is the Census Transform Good for Robust Optic Flow Computation? , 2013, SSVM.

[18]  Rudy Lauwereins,et al.  Robust stereo matching with fast Normalized Cross-Correlation over shape-adaptive regions , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[19]  Minh N. Do,et al.  Joint Histogram-Based Cost Aggregation for Stereo Matching , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Richard Szeliski,et al.  High-accuracy stereo depth maps using structured light , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[21]  Trevor Darrell,et al.  Hierarchical Discrete Distribution Decomposition for Match Density Estimation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Xiao Lu,et al.  Robust stereo matching with trinary cross color census and triple image-based refinements , 2017, EURASIP J. Adv. Signal Process..

[23]  Pierre Boulanger,et al.  Radiometric invariant stereo matching based on relative gradients , 2012, 2012 19th IEEE International Conference on Image Processing.

[24]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[25]  Petros Daras,et al.  Enhanced disparity estimation in stereo images , 2015, Image Vis. Comput..

[26]  Guangjun Xie,et al.  An Efficient Stereo Matching Algorithm Based on Four-Moded Census Transform for High-Resolution Images , 2018 .

[27]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[28]  Cheolkon Jung,et al.  Variational Fusion of Time-of-Flight and Stereo Data for Depth Estimation Using Edge-Selective Joint Filtering , 2018, IEEE Transactions on Multimedia.

[29]  Loris Nanni,et al.  Local binary patterns variants as texture descriptors for medical image analysis , 2010, Artif. Intell. Medicine.

[30]  Seungryong Kim,et al.  Mahalanobis Distance Cross-Correlation for Illumination-Invariant Stereo Matching , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[31]  Sos S. Agaian,et al.  Non-Linear Direct Multi-Scale Image Enhancement Based on the Luminance and Contrast Masking Characteristics of the Human Visual System , 2013, IEEE Transactions on Image Processing.

[32]  Kai Wang,et al.  Pixel to Patch Sampling Structure and Local Neighboring Intensity Relationship Patterns for Texture Classification , 2013, IEEE Signal Processing Letters.

[33]  Xing Mei,et al.  On building an accurate stereo matching system on graphics hardware , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[34]  Qingxiong Yang,et al.  A non-local cost aggregation method for stereo matching , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Truong Q. Nguyen,et al.  Local Disparity Estimation With Three-Moded Cross Census and Advanced Support Weight , 2013, IEEE Transactions on Multimedia.

[36]  Jianfei Cai,et al.  LETRIST: Locally Encoded Transform Feature Histogram for Rotation-Invariant Texture Classification , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[37]  Jinxiang Wang,et al.  Stereo Matching with Improved Radiometric Invariant Matching Cost and Disparity Refinement , 2016, ICIC.

[38]  Andreas Geiger,et al.  Object scene flow for autonomous vehicles , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Ramin Zabih,et al.  Non-parametric Local Transforms for Computing Visual Correspondence , 1994, ECCV.

[40]  Xiaoyang Tan,et al.  Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions , 2007, IEEE Transactions on Image Processing.

[41]  Heiko Hirschmüller,et al.  Evaluation of Cost Functions for Stereo Matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.