Robust and Unbiased Foreground / Background Energy for Multi-view Stereo

This paper revisits the graph-cuts based approach for solving the multi-view stereo problem, and proposes a novel foreground / background energy which is shown to be unbiased and robust against noisy depth maps. Unlike most existing works which focus on deriving a robust photo-consistency energy, this paper targets at deriving a robust and unbiased foreground / background energy. By introducing a novel data-dependent foreground / background energy, we show that it is possible to recover the object surface from noisy depth maps even in the absence of the photo-consistency energy. This demonstrates that the foreground / background energy is equally important as the photo-consistency energy in graph-cuts based methods. Experiments on real data sequences further show that high quality reconstructions can be achieved using our proposed foreground / background energy with a very simple photo-consistency energy.

[1]  Roberto Cipolla,et al.  Multiview Stereo via Volumetric Graph-Cuts and Occlusion Robust Photo-Consistency , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[3]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[4]  Victor S. Lempitsky,et al.  Oriented Visibility for Multiview Reconstruction , 2006, ECCV.

[5]  Roberto Cipolla,et al.  Multi-view stereo via volumetric graph-cuts , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[6]  Nassir Navab,et al.  Multi-View Reconstruction using Narrow-Band Graph-Cuts and Surface Normal Optimization , 2008, BMVC.

[7]  Francis Schmitt,et al.  Silhouette and stereo fusion for 3D object modeling , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[8]  Vladimir Kolmogorov,et al.  Multi-camera Scene Reconstruction via Graph Cuts , 2002, ECCV.

[9]  Gabriel Taubin,et al.  A signal processing approach to fair surface design , 1995, SIGGRAPH.

[10]  Marc Pollefeys,et al.  Multi-view reconstruction using photo-consistency and exact silhouette constraints: a maximum-flow formulation , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[11]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Roberto Cipolla,et al.  Probabilistic visibility for multi-view stereo , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Larry S. Davis,et al.  3D Surface Reconstruction Using Graph Cuts with Surface Constraints , 2006, ECCV.