Consistent Stereo Matching Under Varying Radiometric Conditions

A consistent stereo matching (CSM) algorithm under varying radiometric conditions, such as lighting and exposure variations, for intermediate view synthesis is proposed in this work. First, we transform the colors of stereo images adaptively so that they are similar at corresponding pixels. Since the correspondences are generally unknown before stereo matching, we estimate pseudo-disparity vectors by sorting pixels based on the cumulative color histograms and use those pseudo vectors in the color transform. Then, to improve the accuracy of stereo matching, we jointly estimate the disparity maps for virtual intermediate views as well as those for real views, based on the consistency criterion that an object point should have the same disparity through all the views. Specifically, we compute matching costs using the reliability term and aggregate the costs to obtain initial disparity maps. We then refine the initial disparity maps by minimizing an energy function, which includes the consistency term. Experimental results show that the proposed CSM algorithm significantly reduces the error rate of disparity estimation under different radiometric conditions and synthesizes high quality intermediate views.

[1]  Jean Ponce,et al.  Computer Vision: A Modern Approach , 2002 .

[2]  Andrew Zisserman,et al.  Wide baseline stereo matching , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[3]  E. Land.  The retinex theory of color vision. , 1977, Scientific American.

[4]  Janusz Konrad,et al.  Robust quadtree-based disparity estimation for the reconstruction of intermediate stereoscopic images , 1998, Electronic Imaging.

[5]  Jeffrey S. McVeigh,et al.  Intermediate view synthesis considering occluded and ambiguously referenced image regions , 1996, Signal Process. Image Commun..

[6]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[7]  Pascal Fua,et al.  Combining Stereo and Monocular Information to Compute Dense Depth Maps that Preserve Depth Discontinuities , 1991, IJCAI.

[8]  Heiko Hirschmüller,et al.  Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Joost van de Weijer,et al.  Computational Color Constancy: Survey and Experiments , 2011, IEEE Transactions on Image Processing.

[10]  T. Poggio,et al.  A generalized ordering constraint for stereo correspondence , 1984 .

[11]  Joost van de Weijer,et al.  Author Manuscript, Published in "ieee Transactions on Image Processing Edge-based Color Constancy , 2022 .

[12]  Graham D. Finlayson,et al.  Shades of Gray and Colour Constancy , 2004, CIC.

[13]  In-So Kweon,et al.  Adaptive Support-Weight Approach for Correspondence Search , 2006, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Nanning Zheng,et al.  Stereo Matching Using Belief Propagation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Narendra Ahuja,et al.  Matching Two Perspective Views , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Sing Bing Kang,et al.  Stereo for Image-Based Rendering using Image Over-Segmentation , 2007, International Journal of Computer Vision.

[17]  Bernt Schiele,et al.  Comprehensive Colour Image Normalization , 1998, ECCV.

[18]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[19]  Geoffrey Egnal,et al.  A stereo confidence metric using single view imagery with comparison to five alternative approaches , 2004, Image Vis. Comput..

[20]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  D Marr,et al.  Cooperative computation of stereo disparity. , 1976, Science.

[22]  Brian V. Funt,et al.  Color Constant Color Indexing , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Chang-Su Kim,et al.  Image Enhancement using Sorted Histogram Specification and POCS Postprocessing , 2007, 2007 IEEE International Conference on Image Processing.

[24]  Richard Szeliski,et al.  High-accuracy stereo depth maps using structured light , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[25]  Carlo Tomasi,et al.  Surfaces with occlusions from layered stereo , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Sang Uk Lee,et al.  Robust Stereo Matching Using Adaptive Normalized Cross-Correlation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Andreas Klaus,et al.  Segment-Based Stereo Matching Using Belief Propagation and a Self-Adapting Dissimilarity Measure , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[28]  Hujun Bao,et al.  Consistent Depth Maps Recovery from a Video Sequence , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Takeo Kanade,et al.  Virtualized Reality: Constructing Virtual Worlds from Real Scenes , 1997, IEEE Multim..

[30]  Jian Sun,et al.  Symmetric stereo matching for occlusion handling , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[31]  Narendra Ahuja,et al.  Two-view Matching , 1988, [1988 Proceedings] Second International Conference on Computer Vision.

[32]  Panos Trahanias,et al.  Color image enhancement through 3-D histogram equalization , 1992, Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol. III. Conference C: Image, Speech and Signal Analysis,.

[33]  G. Buchsbaum.  A spatial processor model for object colour perception , 1980 .

[34]  Minglun Gong,et al.  Fast stereo matching using reliability-based dynamic programming and consistency constraints , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[35]  Federico Tombari,et al.  Classification and evaluation of cost aggregation methods for stereo correspondence , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Heiko Hirschmüller,et al.  Evaluation of Stereo Matching Costs on Images with Radiometric Differences , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Janusz Konrad,et al.  Occlusion-Aware View Interpolation , 2008, EURASIP J. Image Video Process..

[38]  Gauthier Lafruit,et al.  High-Speed Stream-Centric Dense Stereo and View Synthesis on Graphics Hardware , 2007, 2007 IEEE 9th Workshop on Multimedia Signal Processing.

[39]  Ruigang Yang,et al.  Real-time consensus-based scene reconstruction using commodity graphics hardware , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[40]  Franz Franchetti,et al.  High Performance Stereo Vision Designed for Massively Data Parallel Platforms , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[41]  Christopher Joseph Pal,et al.  Learning Conditional Random Fields for Stereo , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Dorin Comaniciu,et al.  Kernel-Based Object Tracking , 2003, IEEE Trans. Pattern Anal. Mach. Intell..