Visual modeling : from images to images

This paper contains two parts. In the first part an automatic processing pipeline is presented that analyses an image sequence and automatically extracts camera motion, calibration and scene geometry. The system combines state-of-the-art algorithms developed in computer vision, computer graphics and photogrammetry. The approach consists of two stages. Salient features are extracted and tracked throughout the sequence to compute the camera motion and calibration and the 3D structure of the observed features. Then a dense estimate of the surface geometry of the observed scene is computed using stereo matching. The second part of the paper discusses how this information can be used for visualization. Traditionally, a textured 3D model is constructed from the computed information and used to render new images. Alternatively, it is also possible to avoid the need for an explicit 3D model and to obtain new views directly by combining the appropriate pixels from recorded views. It is interesting to note that even when there is an ambiguity on the reconstructed geometry, correct new images can often still be generated.

[1]  Reinhard Koch,et al.  Hand-held acquisition of 3D models with a video camera , 1999, Second International Conference on 3-D Digital Imaging and Modeling (Cat. No.PR00062).

[2]  Ingemar J. Cox,et al.  A Maximum Likelihood Stereo Algorithm , 1996, Comput. Vis. Image Underst..

[3]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[4]  Luc Van Gool,et al.  Stratified Self-Calibration with the Modulus Constraint , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Reinhard Koch,et al.  Automated reconstruction of 3D scenes from sequences of images , 2000 .

[6]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[7]  Reinhard Koch,et al.  Calibration of hand-held camera sequences for plenoptic modeling , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[8]  Reinhard Koch,et al.  A simple and efficient rectification method for general motion , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[9]  S. Shankar Sastry,et al.  Euclidean Reconstruction and Reprojection Up to Subgroups , 2004, International Journal of Computer Vision.

[10]  KanadeTakeo,et al.  Shape and motion from image streams under orthography , 1992 .

[11]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[12]  William E. Lorensen,et al.  Decimation of triangle meshes , 1992, SIGGRAPH.

[13]  Luc Van Gool,et al.  Some geometric insight in self-calibration and critical motion sequences , 2000 .

[14]  Peter F. Sturm,et al.  Critical motion sequences for the self-calibration of cameras and stereo systems with variable focal length , 1999, Image Vis. Comput..

[15]  Reinhard Koch,et al.  An automatic method for acquiring 3D models from photographs : applications to an archaeological site , 1999 .

[16]  Highlight and Re ection-Independent Multiresolution Textures from Image Sequences , 1997 .

[17]  Rajiv Gupta,et al.  Stereo from uncalibrated cameras , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  Reinhard Koch,et al.  A Geometric Approach to Lightfield Calibration , 1999, CAIP.

[19]  Yizhou Yu,et al.  Efficient View-Dependent Image-Based Rendering with Projective Texture-Mapping , 1998, Rendering Techniques.

[20]  Reinhard Koch,et al.  Multi Viewpoint Stereo from Uncalibrated Video Sequences , 1998, ECCV.

[21]  Olivier D. Faugeras,et al.  What can be seen in three dimensions with an uncalibrated stereo rig , 1992, ECCV.

[22]  Bill Triggs,et al.  Critical Motions for Auto-Calibration When Some Intrinsic Parameters Can Vary , 2000, Journal of Mathematical Imaging and Vision.

[23]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[24]  O. D. Faugeras,et al.  Camera Self-Calibration: Theory and Experiments , 1992, ECCV.

[25]  Peter F. Sturm,et al.  Critical motion sequences for monocular self-calibration and uncalibrated Euclidean reconstruction , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  David Salesin,et al.  Surface light fields for 3D photography , 2000, SIGGRAPH.

[27]  Paul A. Beardsley,et al.  Sequential Updating of Projective and Affine Structure from Motion , 1997, International Journal of Computer Vision.

[28]  Luc Van Gool,et al.  Do ambiguous reconstructions always give ambiguous images? , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[29]  Lutz Falkenhagen.  Hierarchical Block-Based Disparity Estimation Considering Neighbourhood Constraints , 1997 .

[30]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[31]  Andrew Zisserman,et al.  Applications of Invariance in Computer Vision , 1993, Lecture Notes in Computer Science.

[32]  Reinhard Koch,et al.  Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[33]  Maarten Vergauwen,et al.  Virtual Models from Video and Vice-Versa , 2001 .

[34]  Maarten Vergauwen,et al.  A Hierarchical Symmetric Stereo Algorithm Using Dynamic Programming , 2002, International Journal of Computer Vision.

[35]  Reinhard Koch,et al.  Plenoptic Modeling and Rendering from Image Sequences Taken by Hand-Held Camera , 1999, DAGM-Symposium.