Visual modelling: from images to images

This paper contains two parts. In the first part an automatic processing pipeline is presented that analyses an image sequence and automatically extracts camera motion, calibration and scene geometry. The system combines state-of-the-art algorithms developed in computer vision, computer graphics and photogrammetry. The approach consists of two stages. Salient features are extracted and tracked throughout the sequence to compute the camera motion and calibration and the 3D structure of the observed features. Then a dense estimate of the surface geometry of the observed scene is computed using stereo matching. The second part of the paper discusses how this information can be used for visualization. Traditionally, a textured 3D model is constructed from the computed information and used to render new images. Alternatively, it is also possible to avoid the need for an explicit 3D model and to obtain new views directly by combining the appropriate pixels from recorded views. It is interesting to note that even when there is an ambiguity on the reconstructed geometry, correct new images can often still be generated. Copyright © 2002 John Wiley & Sons, Ltd.

[1]  Reinhard Koch,et al.  A Geometric Approach to Lightfield Calibration , 1999, CAIP.

[2]  Bill Triggs,et al.  Critical Motions for Auto-Calibration When Some Intrinsic Parameters Can Vary , 2000, Journal of Mathematical Imaging and Vision.

[3]  Paul A. Beardsley,et al.  Sequential Updating of Projective and Affine Structure from Motion , 1997, International Journal of Computer Vision.

[4]  Reinhard Koch,et al.  Hand-held acquisition of 3D models with a video camera , 1999, Second International Conference on 3-D Digital Imaging and Modeling (Cat. No.PR00062).

[5]  Rajiv Gupta,et al.  Stereo from uncalibrated cameras , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Maarten Vergauwen,et al.  Virtual Models from Video and Vice-Versa , 2001 .

[7]  David Salesin,et al.  Surface light fields for 3D photography , 2000, SIGGRAPH.

[8]  Luc Van Gool,et al.  Do ambiguous reconstructions always give ambiguous images? , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[9]  Reinhard Koch,et al.  An automatic method for acquiring 3D models from photographs : applications to an archaeological site , 1999 .

[10]  Reinhard Koch,et al.  Multi Viewpoint Stereo from Uncalibrated Video Sequences , 1998, ECCV.

[11]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[12]  Yizhou Yu,et al.  Efficient View-Dependent Image-Based Rendering with Projective Texture-Mapping , 1998, Rendering Techniques.

[13]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[14]  Reinhard Koch,et al.  Image-Based Rendering from Uncalibrated Lightfields with Scalable Geometry , 2000, Theoretical Foundations of Computer Vision.

[15]  Michael Bosse,et al.  Unstructured lumigraph rendering , 2001, SIGGRAPH.

[16]  Reinhard Koch,et al.  Plenoptic Modeling and Rendering from Image Sequences Taken by Hand-Held Camera , 1999, DAGM-Symposium.

[17]  Reinhard Koch,et al.  Self-Calibration and Metric Reconstruction Inspite of Varying and Unknown Intrinsic Camera Parameters , 1999, International Journal of Computer Vision.

[18]  Michael Werman,et al.  Multiresolution Textures from Image Sequences , 1997, IEEE Computer Graphics and Applications.

[19]  Bill Triggs,et al.  Autocalibration and the absolute quadric , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[21]  Ingemar J. Cox,et al.  A Maximum Likelihood Stereo Algorithm , 1996, Comput. Vis. Image Underst..

[22]  Olivier D. Faugeras,et al.  What can be seen in three dimensions with an uncalibrated stereo rig , 1992, ECCV.

[23]  O. D. Faugeras,et al.  Camera Self-Calibration: Theory and Experiments , 1992, ECCV.

[24]  Maarten Vergauwen,et al.  A Hierarchical Symmetric Stereo Algorithm Using Dynamic Programming , 2002, International Journal of Computer Vision.

[25]  Peter Sturm.  Critical motion sequences for the self-calibration of cameras and stereo systems with variable focal length , 2002, Image Vis. Comput..

[26]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[27]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Peter F. Sturm,et al.  Critical motion sequences for monocular self-calibration and uncalibrated Euclidean reconstruction , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[29]  Reinhard Koch,et al.  A simple and efficient rectification method for general motion , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[30]  Reinhard Koch,et al.  Calibration of hand-held camera sequences for plenoptic modeling , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[31]  Harry Shum,et al.  Plenoptic sampling , 2000, SIGGRAPH.

[32]  Reinhard Koch,et al.  Automated reconstruction of 3D scenes from sequences of images , 2000 .

[33]  Luc Van Gool,et al.  Some geometric insight in self-calibration and critical motion sequences , 2000 .