Multi-Resolution Disparity Processing and Fusion for Large High-Resolution Stereo Image

Large panoramic views with high resolution have the advantage of a wide field of view over regular stereo views. However, the large size and high resolution impose difficulties on the stereo matching problem such as complexity and structure ambiguity, respectively. In this paper, effective multi-resolution disparity processing to resolve the difficulties is presented. We propose to adaptively determine the disparity search range based on the combined local structure from image intensity and initial disparity. The adaptive disparity range is able to propagate the smoothness property at low resolution to high resolution while preserving fine details. It reduces structure ambiguity as well as computational complexity. To reduce the disparity quantization error at the coarse level, we propose a reliable multiple fitting algorithm that is noticeably effective on the round surface. The spatial-multi-resolution total variation method is investigated to minimize inconsistency in space-scale dimension . The experimental results on the Middlebury datasets and real-world high-resolution images demonstrate that the proposed multi-resolution scheme produces high-quality and high-resolution disparity maps by fusing individual multi-scale disparity maps, while reducing complexity.

[1]  Jan-Michael Frahm,et al.  Adaptive Scale Selection for Hierarchical Stereo , 2011, BMVC.

[2]  Carsten Rother,et al.  Fast cost-volume filtering for visual correspondence and beyond , 2011, CVPR 2011.

[3]  Joost van de Weijer,et al.  Adaptive Structure Tensors and their Applications , 2006, Visualization and Processing of Tensor Fields.

[4]  Dani Lischinski,et al.  Joint bilateral upsampling , 2007, SIGGRAPH 2007.

[5]  Jiang Yu Zheng,et al.  Scanning Depth of Route Panorama Based on Stationary Blur , 2008, International Journal of Computer Vision.

[6]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  M. Vergauwen,et al.  A hierarchical stereo algorithm using dynamic programming , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[8]  Thomas A. DeFanti,et al.  Acquisition of stereo panoramas for display in VR environments , 2011, Electronic Imaging.

[9]  Björn E. Ottersten,et al.  Pixel weighted average strategy for depth sensor data fusion , 2010, 2010 IEEE International Conference on Image Processing.

[10]  Daniele D. Giusto,et al.  Hierarchical block matching for disparity estimation in stereo sequences , 1995, Proceedings., International Conference on Image Processing.

[11]  Sebastian Thrun,et al.  A Noise‐aware Filter for Real‐time Depth Upsampling , 2008 .

[12]  Richard P. Wildes,et al.  Efficient Stereo with Accurate 3-D Boundaries , 2006, BMVC.

[13]  Hao Tang,et al.  Content-Based 3-D Mosaics for Representing Videos of Dynamic Urban Scenes , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  Naim Dahnoun,et al.  Local stereo disparity estimation with novel cost aggregation for sub-pixel accuracy improvement in automotive applications , 2012, 2012 IEEE Intelligent Vehicles Symposium.

[15]  Truong Q. Nguyen,et al.  Local Disparity Estimation With Three-Moded Cross Census and Advanced Support Weight , 2013, IEEE Transactions on Multimedia.

[16]  Truong Q. Nguyen,et al.  Hierarchical depth processing with adaptive search range and fusion , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[17]  Tony Lindeberg,et al.  Scale-Space Theory in Computer Vision , 1993, Lecture Notes in Computer Science.

[18]  Pedro F. Felzenszwalb,et al.  Efficient belief propagation for early vision , 2004, CVPR 2004.

[19]  M. Shimizu,et al.  Precise sub-pixel estimation on area-based matching , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[20]  Truong Q. Nguyen,et al.  An Augmented Lagrangian Method for Total Variation Video Restoration , 2011, IEEE Transactions on Image Processing.

[21]  K. Shirai,et al.  Hierarchical Stereo Matching Via Color Segmentation , 2006, 2006 IEEE 12th Digital Signal Processing Workshop & 4th IEEE Signal Processing Education Workshop.

[22]  Alan L. Yuille,et al.  Multilevel Enhancement and Detection of Stereo Disparity Surfaces , 1995, Artif. Intell..

[23]  Richard P. Wildes,et al.  Coarse-to-fine stereo vision with accurate 3D boundaries , 2010, Image Vis. Comput..

[24]  Miao Liao,et al.  Real-time Global Stereo Matching Using Hierarchical Belief Propagation , 2006, BMVC.

[25]  Wilhelm Burger,et al.  Digital Image Processing - An Algorithmic Introduction using Java , 2008, Texts in Computer Science.

[26]  Larry H. Matthies,et al.  Attenuating stereo pixel-locking via affine window adaptation , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[27]  Mikhail Sizintsev Hierarchical Stereo with Thin Structures and Transparency , 2008, 2008 Canadian Conference on Computer and Robot Vision.

[28]  Shmuel Peleg,et al.  Minimal Aspect Distortion (MAD) Mosaicing of Long Scenes , 2008, International Journal of Computer Vision.

[29]  Ruigang Yang,et al.  Spatial-Depth Super Resolution for Range Images , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[31]  In-So Kweon,et al.  Adaptive Support-Weight Approach for Correspondence Search , 2006, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Vamshhi Pavan Kumar Varma Vegeshna,et al.  Stereo Matching with Color-Weighted Correlation, Hierachical Belief Propagation and Occlusion Handling , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[33]  Chandra Kambhamettu,et al.  Hierarchical belief propagation to reduce search space using CUDA for stereo and motion estimation , 2009, 2009 Workshop on Applications of Computer Vision (WACV).