A New Fusion Algorithm for Depth Images Based on Virtual Views

Common depth image fusion methods use each original image as a reference plane and fuse the depth images using mutual projection. These methods can eliminate inconsistency between the depth images, but they cannot alleviate the point cloud redundancy and computational complexity. This article proposes a virtual view method for depth image fusion, defines a limited number of virtual views by means of view clustering, reduces the redundant calculations, and covers all scenes as much as possible. The depth image is merged ray by ray, and a reliable depth value is obtained via the F-test. Compared with the modified semiglobal matching (TSGM) stereo dense matching algorithm, the accuracy is improved by approximately 50% and the roughness is improved by approximately 50%. Compared with the classic surface reconstruction (SURE) fusion algorithm, there is more fusion depth value in each ray, and the accuracy and roughness are slightly improved. In addition, the algorithm of this article greatly reduces the number of reference planes.

[1]  Shuhan Shen,et al.  Accurate Multiple View 3D Reconstruction Using Patch-Based Stereo for Large-Scale Scenes , 2013, IEEE Transactions on Image Processing.

[2]  Daniel G. Aliaga,et al.  Building reconstruction using manhattan-world grammars , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[3]  Francis Schmitt,et al.  Silhouette and stereo fusion for 3D object modeling , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[4]  Vladimir Kolmogorov,et al.  Multi-camera Scene Reconstruction via Graph Cuts , 2002, ECCV.

[5]  M. Goesele,et al.  Fusion of depth maps with multiple scales , 2011, ACM Trans. Graph..

[6]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[7]  M. Rothermel,et al.  SURE : PHOTOGRAMMETRIC SURFACE RECONSTRUCTION FROM IMAGER Y , 2013 .

[8]  Michael Goesele,et al.  Multi-View Stereo for Community Photo Collections , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[9]  Jan-Michael Frahm,et al.  Efficient and Scalable Depthmap Fusion , 2012, BMVC.

[10]  Leif Kobbelt,et al.  A Surface-Growing Approach to Multi-View Stereo Reconstruction , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Li Yan,et al.  OSSIM: An Object-Based Multiview Stereo Algorithm Using SSIM Index Matching Cost , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Richard Szeliski,et al.  Manhattan-world stereo , 2009, CVPR.

[13]  Xiaoyan Hu,et al.  Least Commitment, Viewpoint-Based, Multi-view Stereo , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[14]  Jiann-Yeou Rau,et al.  Analysis of Oblique Aerial Images for Land Cover and Point Cloud Classification in an Urban Environment , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Michael Goesele,et al.  Multi-View Stereo Revisited , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[16]  Takeo Kanade,et al.  Constructing virtual worlds using dense stereo , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[17]  Reinhard Koch,et al.  Metric 3D Surface Reconstruction from Uncalibrated Image Sequences , 1998, SMILE.

[18]  Jie Li,et al.  Hierarchical and Adaptive Phase Correlation for Precise Disparity Estimation of UAV Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[19]  Roberto Cipolla,et al.  Using Multiple Hypotheses to Improve Depth-Maps for Multi-View Stereo , 2008, ECCV.

[20]  Jan Kautz,et al.  PMBP: PatchMatch Belief Propagation for Correspondence Field Estimation , 2014, International Journal of Computer Vision.

[21]  Hendrik P. A. Lensch,et al.  Multi-View Depth Map Estimation With Cross-View Consistency , 2014, BMVC.

[22]  Derek Bradley,et al.  Accurate multi-view reconstruction using robust binocular stereo and surface meshing , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Eric Q. Li,et al.  Bundled depth-map merging for multi-view stereo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  Reinhard Koch,et al.  Visual Modeling with a Hand-Held Camera , 2004, International Journal of Computer Vision.

[25]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.