Minimizing the Reprojection Error in Surface Reconstruction from Images

This paper addresses the problem of image-based surface reconstruction. The main contribution is the computation of the exact derivative of the reprojection error functional. This allows its rigorous minimization via gradient descent surface evolution. The main difficulty has been to correctly take into account the visibility changes that occur when the surface moves. A geometric and analytical study of these changes is presented and used for the computation of derivative. Our analysis shows the strong influence that the movement of the contour generators has on the reprojection error. As a consequence, during the proper minimization of the reprojection error, the contour generators of the surface are automatically moved to their correct location in the images. Therefore, current methods adding additional silhouettes or apparent contour constraints to ensure this alignment can now be understood and justified by a single criterion: the reprojection error.

[1]  Jean Ponce,et al.  Carved Visual Hulls for Image-Based Modeling , 2006, International Journal of Computer Vision.

[2]  Francis Schmitt,et al.  Silhouette and stereo fusion for 3D object modeling , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[3]  Olivier D. Faugeras,et al.  Modelling dynamic scenes by registering multi-view image sequences , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[4]  Stefano Soatto,et al.  Stereoscopic Segmentation , 2003, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[5]  Vladimir Kolmogorov,et al.  Computing geodesics and minimal surfaces via graph cuts , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[6]  S. Osher,et al.  Algorithms Based on Hamilton-Jacobi Formulations , 1988 .

[7]  Victor S. Lempitsky,et al.  Oriented Visibility for Multiview Reconstruction , 2006, ECCV.

[8]  Olivier D. Faugeras,et al.  Designing spatially coherent minimizing flows for variational problems based on active contours , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[9]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[10]  Marc Pollefeys,et al.  Multi-view reconstruction using photo-consistency and exact silhouette constraints: a maximum-flow formulation , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[11]  Olivier D. Faugeras,et al.  Complete Dense Stereovision Using Level Set Methods , 1998, ECCV.

[12]  Long Quan,et al.  A Surface Reconstruction Method Using Global Graph Cut Optimization , 2006, International Journal of Computer Vision.

[13]  Roberto Cipolla,et al.  Multi-view stereo via volumetric graph-cuts , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[14]  R. Keriven.  A variational framework for shape from contours — Research Report CERMICS-2002-221 b , 2003 .

[15]  Francis Schmitt,et al.  Silhouette and stereo fusion for 3D object modeling , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[16]  Tianli Yu,et al.  SDG Cut: 3D Reconstruction of Non-lambertian Objects Using Graph Cuts on Surface Distance Grid , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[17]  Niels Chr. Overgaard,et al.  A Geometric Formulation of Gradient Descent for Variational Problems with Moving Surfaces , 2005, Scale-Space.

[18]  S. Osher,et al.  Visibility and its dynamics in a PDE based implicit framework , 2004 .

[19]  Hugues Talbot,et al.  Globally minimal surfaces by continuous maximal flows , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Douglas Lanman,et al.  Beyond Silhouettes: Surface Reconstruction Using Multi-Flash Photography , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[21]  Ross T. Whitaker,et al.  A Level-Set Approach to 3D Reconstruction from Range Data , 1998, International Journal of Computer Vision.

[22]  C. Strecha,et al.  Wide-baseline stereo from multiple views: A probabilistic account , 2004, CVPR 2004.

[23]  Stefano Soatto,et al.  Tales of shape and radiance in multiview stereo , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[24]  L. Hörmander.  The analysis of linear partial differential operators , 1990 .

[25]  Marcus A. Magnor,et al.  Weighted Minimal Hypersurfaces and Their Applications in Computer Vision , 2004, ECCV.

[26]  Marcus A. Magnor,et al.  Weighted Minimal Hypersurface Reconstruction , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Ronald Fedkiw,et al.  Level set methods and dynamic implicit surfaces , 2002, Applied mathematical sciences.

[28]  J. Sethian,et al.  Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulations , 1988 .

[29]  Jan J. Koenderink,et al.  Solid shape , 1990 .

[30]  L. Hörmander,et al.  The Analysis of Linear Partial Differential Operators I: Distribution Theory and Fourier Analysis , 1983 .