3D reconstruction of natural scenes with view-adaptive multi-texturing

We present a 3D reconstruction and modeling system that operates on a number of input photographs that show a natural scene. Approaches from computer graphics and image processing are combined and performance is shown via experiments. Furthermore, reconstruction quality is analyzed w.r.t. the number and distribution of textures, used for reconstruction. The reconstruction pipeline starts with image acquisition, which consists of a number of photographs of the scene that are sequentially taken at different positions. Since the photographs are not acquired concurrently, they are influenced by different illumination conditions that we mandate to be preserved in the final 3D representation. In the second step, object segmentation is applied and camera calibration provided. This allows the application of shape-from-silhouette approaches, namely a hierarchical voxel approach, where different resolution layers are organized within an octree structure. For applying texture mapping, the voxel model is transformed into a wireframe, which provides smoothing of the object's surface and also reduces the number of surface primitives. Finally, a subset of original images is mapped onto the 3D geometry to provide texture information. Here, view-adaptive multi-texturing is used to preserve natural illumination. Intermediate views are interpolated automatically using adaptive real-time weight calculations for original textures.

[1]  Richard Szeliski,et al.  The geometry-image representation tradeoff for rendering , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[2]  Steven M. Seitz,et al.  Photorealistic Scene Reconstruction by Voxel Coloring , 1997, International Journal of Computer Vision.

[3]  Peter Eisert,et al.  Multi-hypothesis, volumetric reconstruction of 3-D objects from multiple calibrated camera views , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[4]  Michael Bosse,et al.  Unstructured lumigraph rendering , 2001, SIGGRAPH.

[5]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[6]  Wei-Chao Chen,et al.  Light field mapping: efficient representation and hardware rendering of surface light fields , 2002, SIGGRAPH.

[7]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[8]  Li Zhang,et al.  Generating a 3D model of a Bayon tower using non-metric imagery , 2001 .

[9]  Richard Szeliski,et al.  Rapid octree construction from image sequences , 1993 .

[10]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[11]  Wojciech Matusik,et al.  Opacity light fields: interactive rendering of surface light fields with view-dependent opacity , 2003, I3D '03.