Parameterized image varieties: a novel approach to the analysis and synthesis of image sequences

This paper addresses the problem of characterizing the space formed by all images of a rigid set of n points observed by a weak perspective or paraperspective camera. By taking explicitly into account the Euclidean constraints associated with calibrated cameras, we show that this space is a six-dimensional variety embedded in R/sup 2n/, and parameterize it using the image positions of three reference points. This parameterization is constructed via linear least squares from point correspondences established across a sequence of images, and it is used to synthesize new pictures without any explicit three-dimensional model. Degenerate scene and camera configurations are analyzed, and experiments with real image sequences are presented.

[1]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[2]  Mi-Suen Lee,et al.  View Synthesis from Unregistered 2-D Images , 1996, Graphics Interface.

[3]  Shenchang Eric Chen,et al.  QuickTime VR: an image-based approach to virtual environment navigation , 1995, SIGGRAPH.

[4]  Mi-Suen Lee,et al.  Synthesizing Novel Views from Unregistered 2‐D Images , 1997, Comput. Graph. Forum.

[5]  Dinesh Manocha,et al.  Computing selected solutions of polynomial equations , 1994, ISSAC '94.

[6]  F. S. Macaulay,et al.  The Algebraic Theory of Modular Systems , 1972 .

[7]  Toshiyuki Sakai,et al.  Obtaining Surface Orientation from Texels under Perspective Projection , 1981, IJCAI.

[8]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[9]  Steven M. Seitz,et al.  Physically-valid view synthesis by image interpolation , 1995, Proceedings IEEE Workshop on Representation of Visual Scenes (In Conjunction with ICCV'95).

[10]  Rajiv Gupta,et al.  Stereo from uncalibrated cameras , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Yiannis Aloimonos,et al.  Perspective approximations , 1990, Image Vis. Comput..

[12]  Olivier D. Faugeras,et al.  What can be seen in three dimensions with an uncalibrated stereo rig , 1992, ECCV.

[13]  Dinesh Manocha,et al.  Algorithm for implicitizing rational parametric surfaces , 1992, Comput. Aided Geom. Des..

[14]  Daphna Weinshall,et al.  Linear and incremental acquisition of invariant shape models from image sequences , 1993, 1993 (4th) International Conference on Computer Vision.

[15]  Carlo Tomasi.  Pictures and trails: a new framework for the computation of shape and motion from perspective image sequences , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Eamon B. Barrett,et al.  Some invariant linear methods in photogrammetry and model-matching , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Kiriakos N. Kutulakos,et al.  Non-Euclidean Object Representations for Calibration-Free Video Overlay , 1996, Object Representation in Computer Vision.

[18]  J J Koenderink,et al.  Affine structure from motion. , 1991, Journal of the Optical Society of America. A, Optics and image science.

[19]  Takeo Kanade,et al.  A Paraperspective Factorization Method for Shape and Motion Recovery , 1994, ECCV.

[20]  Olivier D. Faugeras,et al.  3-D scene representation as a collection of images , 1994, Proceedings of 12th International Conference on Pattern Recognition.