Stochastic refinement of the visual hull to satisfy photometric and silhouette consistency constraints

An iterative method for reconstructing a 3D polygonal mesh and color texture map from multiple views of an object is presented. In each iteration, the method first estimates a texture map given the current shape estimate. The texture map and its associated residual error image are obtained via maximum a posteriori estimation and reprojection of the multiple views into texture space. Next, the surface shape is adjusted to minimize residual error in texture space. The surface is deformed towards a photometrically-consistent solution via a series of 1D epipolar searches at randomly selected surface points. The texture space formulation has improved computational complexity over standard image-based error approaches, and allows computation of the reprojection error and uncertainty for any point on the surface. Moreover, shape adjustments can be constrained such that the recovered model's silhouette matches those of the input images. Experiments with real world imagery demonstrate the validity of the approach.

[1]  John Porrill,et al.  Curve matching and stereo calibration , 1991, Image and Vision Computing.

[2]  Richard Szeliski,et al.  Real-Time Octree Generation from Rotating Objects , 1990 .

[3]  Heinrich Müller,et al.  Spatial free-form deformation with scattered data interpolation methods , 1992, Comput. Graph..

[4]  A. Laurentini,et al.  The Visual Hull Concept for Silhouette-Based Image Understanding , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Michael Garland,et al.  Surface simplification using quadric error metrics , 1997, SIGGRAPH.

[6]  Andrew W. Fitzgibbon,et al.  Automatic 3D Model Construction for Turn-Table Sequences , 1998, SMILE.

[7]  Thomas Malzbender,et al.  Generalized Voxel Coloring , 1999, Workshop on Vision Algorithms.

[8]  Paul A. Viola,et al.  Roxels: responsibility weighted 3D volume reconstruction , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[9]  Kiriakos N. Kutulakos.  Approximate N-View Stereo , 2000, ECCV.

[10]  Ramesh Raskar,et al.  Image-based visual hulls , 2000, SIGGRAPH.

[11]  Victor J. Milenkovic.  Densest translational lattice packing of non-convex polygons (extended abstract) , 2000, SCG '00.

[12]  Latex2e Cmu-techreport Image-based Multiresolution Modeling by Surface Deformation , 2000 .

[13]  A Probabilistic Framework for Space Carving , 2001, ICCV.

[14]  Francis Schmitt,et al.  Multi-stereo 3D object reconstruction , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[15]  T. Matsuyama,et al.  3 D Shape From Multi-viewpoint Images Using Deformable Mesh Model , 2002 .

[16]  Ronald W. Schafer,et al.  Novel volumetric scene reconstruction methods for new view synthesis , 2002 .

[17]  Stan Sclaroff,et al.  Stochastic mesh-based multiview reconstruction , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[18]  Tal Hassner,et al.  What Does the Scene Look Like from a Scene Point? , 2002, ECCV.

[19]  Steven M. Seitz,et al.  Photorealistic Scene Reconstruction by Voxel Coloring , 1997, International Journal of Computer Vision.

[20]  Kiriakos N. Kutulakos,et al.  A Theory of Shape by Space Carving , 2000, International Journal of Computer Vision.