Automatic 3D model acquisition and generation of new images from video sequences

We describe a method to completely automatically recover 3D scene structure together with 3D camera positions from a sequence of images acquired by an unknown camera undergoing unknown movement. Unlike "tuned" systems which use calibration objects or markers to recover this information, and are therefore often limited to a particular scale, the approach of this paper is more general and can be applied to a large class of scenes. It is demonstrated here for interior and exterior sequences using both controlled-motion and handheld cameras. The paper reviews Computer Vision research into structure and motion recovery, providing a tutorial introduction to the geometry of multiple views, estimation and correspondence in video streams. The core method, which simultaneously extracts the 3D scene structure and camera positions, is applied to the automated recovery of VRML 3D textured models from a video sequence.

[1]  Paul A. Beardsley,et al.  3D Model Acquisition from Extended Image Sequences , 1996, ECCV.

[2]  O. Faugeras.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[3]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[5]  Andrew Zisserman,et al.  Robust Parameterization and Computation of the Trifocal Tensor , 1996, BMVC.

[6]  Paul A. Beardsley,et al.  Navigation using Affine Structure from Motion , 1994, ECCV.

[7]  Stéphane Laveau.  Géométrie d'un système de N caméras : théorie, estimation et applications. (Geometry of a System of n Cameras. Theory. Estimation. Applications) , 1996 .

[8]  Amnon Shashua,et al.  Trilinearity in Visual Recognition by Alignment , 1994, ECCV.

[9]  Philip H. S. Torr,et al.  Outlier detection and motion segmentation , 1993, Other Conferences.

[10]  Paul Debevec,et al.  Modeling and Rendering Architecture from Photographs , 1996, SIGGRAPH 1996.

[11]  Cordelia Schmid,et al.  The Geometry and Matching of Curves in Multiple Views , 1998, ECCV.

[12]  Rajiv Gupta,et al.  Stereo from uncalibrated cameras , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  H. C. Longuet-Higgins,et al.  A computer algorithm for reconstructing a scene from two projections , 1981, Nature.

[14]  Reinhard Koch,et al.  Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[15]  Olivier D. Faugeras,et al.  On the geometry and algebra of the point and line correspondences between N images , 1995, Proceedings of IEEE International Conference on Computer Vision.

[16]  Rachid Deriche,et al.  A Robust Technique for Matching two Uncalibrated Images Through the Recovery of the Unknown Epipolar Geometry , 1995, Artif. Intell..

[17]  Soren W. Henriksen,et al.  Manual of photogrammetry , 1980 .

[18]  Andrew Zisserman,et al.  Geometric invariance in computer vision , 1992 .

[19]  Richard I. Hartley,et al.  A linear method for reconstruction from lines and points , 1995, Proceedings of IEEE International Conference on Computer Vision.

[20]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[21]  Olivier D. Faugeras,et al.  What can be seen in three dimensions with an uncalibrated stereo rig , 1992, ECCV.

[22]  O. D. Faugeras,et al.  Camera Self-Calibration: Theory and Experiments , 1992, ECCV.

[23]  KanadeTakeo,et al.  Shape and motion from image streams under orthography , 1992 .

[24]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Andrew W. Fitzgibbon,et al.  Automatic Camera Recovery for Closed or Open Image Sequences , 1998, ECCV.

[26]  Cordelia Schmid,et al.  Automatic line matching across views , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.