A Integrated Depth Fusion Algorithm for Multi-view Stereo

In this paper, we propose a new integrated depth fusion algorithm for multi-view stereo. Starting from an embedding space such as the visual hull, we will first conduct robust 3D depth estimation (represented as 3D points) based on image correlation. Next a volumetric saliency weighted normal vector field is constructed from which a watertight 3D surface can be extracted using the graph cut algorithm. Finally an explicit surface evolution will be conducted to recover the finer geometry details of the recovered shape. The experiments on the benchmark datasets show that our algorithm can obtain high quality reconstruction results that are comparable with the state-of-art methods, with considerable less computational time and complexity.

[1]  J. Kruskal.  On the shortest spanning subtree of a graph and the traveling salesman problem , 1956 .

[2]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[3]  Tony DeRose,et al.  Surface reconstruction from unorganized points , 1992, SIGGRAPH.

[4]  A. Laurentini,et al.  The Visual Hull Concept for Silhouette-Based Image Understanding , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Guillermo Sapiro,et al.  Three Dimensional Object Modeling via Minimal Surfaces , 1996, ECCV.

[6]  S. Osher,et al.  Fast surface reconstruction using the level set method , 2001, Proceedings IEEE Workshop on Variational and Level Set Methods in Computer Vision.

[7]  Francis Schmitt,et al.  Multi-stereo 3D object reconstruction , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[8]  Vladimir Kolmogorov,et al.  Multi-camera Scene Reconstruction via Graph Cuts , 2002, ECCV.

[9]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  O. Faugeras,et al.  Variational principles, surface evolution, PDE's, level set methods and the stereo problem , 1998, 5th IEEE EMBS International Summer School on Biomedical Imaging, 2002..

[11]  Stefano Soatto,et al.  Tales of shape and radiance in multiview stereo , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[12]  Vladimir Kolmogorov,et al.  Computing geodesics and minimal surfaces via graph cuts , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[13]  Francis Schmitt,et al.  Silhouette and stereo fusion for 3D object modeling , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[14]  Stefano Soatto,et al.  Multi-view stereo beyond Lambert , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[15]  Ahmed M. Elgammal,et al.  High Resolution Acquisition, Learning and Transfer of Dynamic 3‐D Facial Expressions , 2004, Comput. Graph. Forum.

[16]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Hong Qin,et al.  Shape Reconstruction from 3D and 2D Data Using PDE-Based Deformable Surfaces , 2004, ECCV.

[18]  Roberto Cipolla,et al.  Multi-view stereo via volumetric graph-cuts , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[19]  Long Quan,et al.  A quasi-dense approach to surface reconstruction from uncalibrated images , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Stefano Soatto,et al.  Multi-View Stereo Reconstruction of Dense Shape and Complex Appearance , 2005, International Journal of Computer Vision.

[21]  Marc Pollefeys,et al.  Multi-view reconstruction using photo-consistency and exact silhouette constraints: a maximum-flow formulation , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[22]  Leif Kobbelt,et al.  Hierarchical Volumetric Multi-view Stereo Reconstruction of Manifold Surfaces based on Dual Graph Embedding , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[23]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[24]  Michael Goesele,et al.  Multi-View Stereo Revisited , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[25]  Jan-Michael Frahm,et al.  Real-Time Visibility-Based Fusion of Depth Maps , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[26]  Roberto Cipolla,et al.  Multiview Stereo via Volumetric Graph-Cuts and Occlusion Robust Photo-Consistency , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Leif Kobbelt,et al.  A Surface-Growing Approach to Multi-View Stereo Reconstruction , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Victor S. Lempitsky,et al.  Global Optimization for Shape Fitting , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Michael Goesele,et al.  Multi-View Stereo for Community Photo Collections , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[30]  J. Ponce,et al.  Accurate, Dense, and Robust Multi-View Stereopsis , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Long Quan,et al.  Image-Based Modeling by Joint Segmentation , 2007, International Journal of Computer Vision.

[32]  Derek Bradley,et al.  Accurate multi-view reconstruction using robust binocular stereo and surface meshing , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Roberto Cipolla,et al.  Using Multiple Hypotheses to Improve Depth-Maps for Multi-View Stereo , 2008, ECCV.

[34]  Jean Ponce,et al.  Carved Visual Hulls for Image-Based Modeling , 2006, International Journal of Computer Vision.

[35]  Qionghai Dai,et al.  Continuous depth estimation for multi-view stereo , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Daniel Cremers,et al.  Anisotropic Minimal Surfaces Integrating Photoconsistency and Normal Information for Multiview Stereo , 2010, ECCV.