Geometry and Texture from Thousands of Images

This paper presents a novel method for automatically recovering dense surface patches using large sets (1000's) of calibrated images taken from arbitrary positions within the scene. Physical instruments, such as Global Positioning System (GPS), inertial sensors, and inclinometers, are used to estimate the position and orientation of each image. Some of the most important characteristics of our approach are that it: (1) uses and refines noisy calibration estimates; (2) compensates for large variations in illumination; (3) tolerates significant soft occlusion (e.g. tree branches); and (4) associates, at a fundamental level, an estimated normal (eliminating the frontal-planar assumption) and texture with each surface patch.

[1]  A. Hanson,et al.  Overview of the SRI Cartographic Modeling Environment , 1992 .

[2]  S. Teller.  Automated urban model acquisition : Project rationale and status , 1999 .

[3]  V. Bove,et al.  Semiautomatic 3D-model extraction from uncalibrated 2D-camera views , 1995 .

[4]  Tomas Lozano-Perez,et al.  Dense Depth Maps from Epipolar Images , 1996 .

[5]  Robert T. Collins,et al.  A space-sweep approach to true multi-image matching , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  De Couto,et al.  Instrumentation for rapidly acquiring pose-imagery , 1998 .

[7]  Xiaoping Li,et al.  Optimal L1L1 Approximation of the Gaussian Kernel With Application to Scale-Space Construction , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[9]  Seth J. Teller,et al.  Acquisition of a large pose-mosaic dataset , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[10]  Roger Y. Tsai,et al.  Multiframe Image Point Matching and 3-D Surface Reconstruction , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Poorvi L. Vora,et al.  Digital color cameras - 2 - Spectral response , 1997 .

[12]  Seth J. Teller,et al.  Toward urban model acquisition from geo-located images , 1998, Proceedings Pacific Graphics '98. Sixth Pacific Conference on Computer Graphics and Applications (Cat. No.98EX208).

[13]  Allen R. Hanson,et al.  System for automated site model acquisition , 1995, Defense, Security, and Sensing.

[14]  William H. Press,et al.  The Art of Scientific Computing Second Edition , 1998 .

[15]  Paul Debevec,et al.  Modeling and Rendering Architecture from Photographs , 1996, SIGGRAPH 1996.

[16]  Satyan R. Coorg,et al.  Pose imagery and automated three-dimensional modeling of urban environments , 1998 .

[17]  Kiriakos N. Kutulakos,et al.  A Theory of Shape by Space Carving , 2000, International Journal of Computer Vision.

[18]  Kiriakos N. Kutulakos,et al.  What Do N Photographs Tell Us about 3D Shape , 1998 .

[19]  Takeo Kanade,et al.  A Multiple-Baseline Stereo , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  D. T. Gering,et al.  Object modeling using tomography and photography , 1999, Proceedings IEEE Workshop on Multi-View Modeling and Analysis of Visual Scenes (MVIEW'99).

[21]  Mei Han,et al.  Interactive construction of 3D models from panoramic mosaics , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[22]  Robert C. Bolles,et al.  Epipolar-plane image analysis: An approach to determining structure from motion , 1987, International Journal of Computer Vision.

[23]  V. Michael Bove,et al.  Semiautomatic 3-D model extraction from uncalibrated 2-D camera views , 1995 .

[24]  J. P. Mellor,et al.  Automatically recovering geometry and texture from large sets of calibrated images , 1999 .

[25]  Richard Szeliski,et al.  Surface modeling with oriented particle systems , 1992, SIGGRAPH.

[26]  G. N. Ramachandran,et al.  Three-dimensional reconstruction from radiographs and electron micrographs: application of convolutions instead of Fourier transforms. , 1971, Proceedings of the National Academy of Sciences of the United States of America.

[27]  SiteCity: a semi-automated site modelling system , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[28]  Steven M. Seitz,et al.  Photorealistic Scene Reconstruction by Voxel Coloring , 1997, International Journal of Computer Vision.

[29]  Allen R. Hanson,et al.  Extracting surface textures and microstructures from multiple aerial images , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.