Realistic surface reconstruction of 3D scenes from uncalibrated image sequences

This contribution addresses the problem of obtaining 3D models from image sequences. A 3D surface description of the scene is extracted completely from a set of uncalibrated camera images of the scene. No prior knowledge about the scene or about the camera is needed to build the 3D models. The only assumptions are the rigidity of the scene objects and opaque object surfaces. The modeling system described here uses a 3-step approach. First, the camera pose and intrinsic parameters are calibrated by tracking salient feature points throughout the sequence. Next, consecutive images of the sequence are treated as stereoscopic image pairs and dense correspondence maps are computed by area matching. Finally, dense and accurate depth maps are computed by linking together all correspondences over the viewpoints. The depth maps are converted to triangular surfaces meshes that are texture mapped for photo-realistic appearance. The feasibility of the approach has been tested on both real and synthetic data and is illustrated here on several outdoor image sequences.

[1]  G. L. Gimel'farb,et al.  Symmetrical approach to the problem of automating stereoscopic measurements in photogrammetry , 1979, CYBERNETICS.

[2]  Michal Irani,et al.  Super resolution from image sequences , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[3]  Gérard G. Medioni,et al.  Object modeling by registration of multiple range images , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[4]  Andreas F. Koschan.  Eine Methodenbank zur Evaluierung von Stereo-Vision-Verfahren , 1991 .

[5]  Richard I. Hartley,et al.  Estimation of Relative Camera Positions for Uncalibrated Cameras , 1992, ECCV.

[6]  Olivier D. Faugeras,et al.  What can be seen in three dimensions with an uncalibrated stereo rig , 1992, ECCV.

[7]  O. D. Faugeras,et al.  Camera Self-Calibration: Theory and Experiments , 1992, ECCV.

[8]  Reinhard Koch,et al.  3-D surface reconstruction from stereoscopic image sequences , 1995, Proceedings of IEEE International Conference on Computer Vision.

[9]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .

[10]  P. Anandan,et al.  Mosaic based representations of video sequences and their applications , 1995, Proceedings of IEEE International Conference on Computer Vision.

[11]  S. P. Mudur,et al.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[12]  Luc Van Gool,et al.  Euclidean 3D Reconstruction from Stereo Sequences with Variable Focal Lengths , 1995, ACCV.

[13]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[14]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[15]  Paul A. Beardsley,et al.  3D Model Acquisition from Extended Image Sequences , 1996, ECCV.

[16]  Ingemar J. Cox,et al.  A Maximum Likelihood Stereo Algorithm , 1996, Comput. Vis. Image Underst..

[17]  Reinhard Koch,et al.  Surface segmentation and modeling of 3D polygonal objects from stereoscopic image pairs , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[18]  Richard Szeliski,et al.  Creating full view panoramic image mosaics and texture-mapped models , 1997, International Conference on Computer Graphics and Interactive Techniques.

[19]  Anders Heyden,et al.  Euclidean reconstruction from image sequences with varying and unknown focal length and principal point , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  Highlight and Re ection-Independent Multiresolution Textures from Image Sequences , 1997 .

[21]  Luc Van Gool,et al.  Self-Calibration from the Absolute Conic on the Plane at Infinity , 1997, CAIP.

[22]  Reinhard Koch,et al.  Multi Viewpoint Stereo from Uncalibrated Video Sequences , 1998, ECCV.

[23]  Mei Han,et al.  Interactive 3D Modeling from Multiple Images Using Scene Regularities , 1998, SMILE.

[24]  Andrew W. Fitzgibbon,et al.  Automatic 3D Model Construction for Turn-Table Sequences , 1998, SMILE.

[25]  Reinhard Koch,et al.  Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[26]  Reinhard Koch,et al.  Robust Calibration and 3D Geometric Modeling From Large Collections of Uncalibrated Images , 1999, DAGM-Symposium.

[27]  S. Teller.  Automated urban model acquisition : Project rationale and status , 1999 .

[28]  Reinhard Koch,et al.  Calibration of hand-held camera sequences for plenoptic modeling , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.