Realistic surface reconstruction of 3D scenes from uncalibrated image sequences

This contribution addresses the problem of obtaining 3D models from image sequences. A 3D surface description of the scene is extracted completely from a set of uncalibrated camera images of the scene. No prior knowledge about the scene or about the camera is needed to build the 3D models. The only assumptions are the rigidity of the scene objects and opaque object surfaces. The modelling system described here uses a three-step approach. First, the camera pose and intrinsic parameters are calibrated by tracking salient feature points throughout the sequence. Next, consecutive images of the sequence are treated as stereoscopic image pairs, and dense correspondence maps are computed by area matching. Finally, dense and accurate depth maps are computed by linking together all correspondences over the viewpoints. The depth maps are converted to triangular surfaces meshes that are texture mapped for photo-realistic appearance. The feasibility of the approach has been tested on both real and synthetic data and is illustrated here on several outdoor image sequences. Copyright © 2000 John Wiley & Sons, Ltd.

[1]  Anders Heyden,et al.  Euclidean reconstruction from image sequences with varying and unknown focal length and principal point , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  Reinhard Koch,et al.  Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[3]  Luc Van Gool,et al.  Euclidean 3D Reconstruction from Image Sequences with Variable Focal Lenghts , 1996, ECCV.

[4]  Luc Van Gool,et al.  Self-Calibration from the Absolute Conic on the Plane at Infinity , 1997, CAIP.

[5]  Reinhard Koch,et al.  Calibration of hand-held camera sequences for plenoptic modeling , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[6]  Olivier D. Faugeras,et al.  What can be seen in three dimensions with an uncalibrated stereo rig , 1992, ECCV.

[7]  O. D. Faugeras,et al.  Camera Self-Calibration: Theory and Experiments , 1992, ECCV.

[8]  Ingemar J. Cox,et al.  A Maximum Likelihood Stereo Algorithm , 1996, Comput. Vis. Image Underst..

[9]  Richard Szeliski,et al.  Creating full view panoramic image mosaics and texture-mapped models , 1997, International Conference on Computer Graphics and Interactive Techniques.

[10]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[11]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .

[12]  Paul A. Beardsley,et al.  3D Model Acquisition from Extended Image Sequences , 1996, ECCV.

[13]  O. Faugeras.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[14]  Richard I. Hartley,et al.  Estimation of Relative Camera Positions for Uncalibrated Cameras , 1992, ECCV.

[15]  Reinhard Koch,et al.  Robust Calibration and 3D Geometric Modeling From Large Collections of Uncalibrated Images , 1999, DAGM-Symposium.

[16]  P. Anandan,et al.  Mosaic based representations of video sequences and their applications , 1995, Proceedings of IEEE International Conference on Computer Vision.

[17]  Reinhard Koch,et al.  3-D surface reconstruction from stereoscopic image sequences , 1995, Proceedings of IEEE International Conference on Computer Vision.

[18]  Mei Han,et al.  Interactive 3D Modeling from Multiple Images Using Scene Regularities , 1998, SMILE.

[19]  Gérard G. Medioni,et al.  Object modelling by registration of multiple range images , 1992, Image Vis. Comput..

[20]  Marc Waelkens,et al.  Computer-aided design and archaeology at Sagalassos: methodology and possibilities of CAD reconstructions of archaeological sites , 1998 .

[21]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[22]  Andrew W. Fitzgibbon,et al.  Automatic 3D Model Construction for Turn-Table Sequences , 1998, SMILE.

[23]  Reinhard Koch,et al.  Surface segmentation and modeling of 3D polygonal objects from stereoscopic image pairs , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[24]  G. L. Gimel'farb,et al.  Symmetrical approach to the problem of automating stereoscopic measurements in photogrammetry , 1979, CYBERNETICS.

[25]  S. Teller.  Automated urban model acquisition : Project rationale and status , 1999 .

[26]  Michael Werman,et al.  Multiresolution Textures from Image Sequences , 1997, IEEE Computer Graphics and Applications.

[27]  Michal Irani,et al.  Super resolution from image sequences , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[28]  Gérard G. Medioni,et al.  Object modeling by registration of multiple range images , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[29]  Reinhard Koch,et al.  Multi Viewpoint Stereo from Uncalibrated Video Sequences , 1998, ECCV.