Image mosaicing for tele-reality applications

This paper presents some techniques for automatically deriving realistic 2-D scenes and 3-D geometric models from video sequences. These techniques can be used to build environments and 3-D models for virtual reality application based on recreating a true scene, i.e., tele-reality applications. The fundamental technique used in this paper is image mosaicing, i.e., the automatic alignment of multiple images into larger aggregates which are then used to represent portions of a 3-D scene. The paper first examines the easiest problems, those of flat scene and panoramic scene mosaicing. It then progresses to more complicated scenes with depth, and concludes with full 3-D models. The paper also discusses a number of novel applications based on tele-reality technology.<<ETX>>

[1]  Olivier D. Faugeras,et al.  What can be seen in three dimensions with an uncalibrated stereo rig , 1992, ECCV.

[2]  L. Quam Hierarchical warp stereo , 1987 .

[3]  William R. Pickering Merging 3-D graphics and imaging: applications and issues , 1993, SIGGRAPH.

[4]  Walter Bender,et al.  Salient video stills: content and context preserved , 1993, MULTIMEDIA '93.

[5]  Lance Williams,et al.  View Interpolation for Image Synthesis , 1993, SIGGRAPH.

[6]  William H. Press,et al.  Book-Review - Numerical Recipes in Pascal - the Art of Scientific Computing , 1989 .

[7]  William H. Press,et al.  The Art of Scientific Computing Second Edition , 1998 .

[8]  Michael J. Black,et al.  Mixture models for optical flow computation , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Carlo H. Séquin,et al.  Adaptive display algorithm for interactive frame rates during visualization of complex virtual environments , 1993, SIGGRAPH.

[10]  Alex Pentland,et al.  Closed-form solutions for physically-based shape modeling and recognition , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Rich Gold,et al.  Ubiquitous computing and augmented reality , 1993, SIGGRAPH.

[12]  Donald P. Greenberg,et al.  Computer Graphics in Architecture , 1974 .

[13]  Sally A. Applin,et al.  The virtual museum: Interactive 3D navigation of a multimedia database , 1992, Comput. Animat. Virtual Worlds.

[14]  Paul S. Heckbert,et al.  Creating Raster Omnimax Images from Multiple Perspective Views Using the Elliptical Weighted Average Filter , 1986, IEEE Computer Graphics and Applications.

[15]  Richard Szeliski,et al.  Surface modeling with oriented particle systems , 1992, SIGGRAPH.

[16]  Alex Pentland,et al.  Recursive estimation of structure and motion using relative orientation constraints , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Michael Mills,et al.  Panoramic overviews for navigating real-world scenes , 1993, MULTIMEDIA '93.

[18]  Lisa M. Brown,et al.  A survey of image registration techniques , 1992, CSUR.

[19]  Andrew Lippman,et al.  Movie-maps: An application of the optical videodisc to computer graphics , 1980, SIGGRAPH '80.

[20]  Richard Szeliski,et al.  Shape from rotation , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  Demetri Terzopoulos,et al.  Physically based models with rigid and deformable components , 1988, IEEE Computer Graphics and Applications.

[22]  John A. Vince,et al.  Virtual reality systems , 1995 .

[23]  GreeneNed Environment mapping and other applications of world projections , 1986 .

[24]  P. Anandan,et al.  Hierarchical Model-Based Motion Estimation , 1992, ECCV.

[25]  Richard Szeliski,et al.  Hierarchical spline-based image registration , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Edward H. Adelson,et al.  Layered representation for motion analysis , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Berthold K. P. Horn Robot vision , 1986, MIT electrical engineering and computer science series.

[28]  Thaddeus Beier,et al.  Feature-based image metamorphosis , 1992, SIGGRAPH.

[29]  Michael Gleicher,et al.  Through-the-lens camera control , 1992, SIGGRAPH.

[30]  Julia B. Schwartz,et al.  From Harper & Row , 1970 .

[31]  Takeo Kanade,et al.  A Multiple-Baseline Stereo , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Steve Mann,et al.  Virtual bellows: constructing high quality stills from video , 1994, Proceedings of 1st International Conference on Image Processing.

[33]  Jack Bresenham Real virtuality: StereoLithography—rapid prototyping in 3-D , 1993, SIGGRAPH.

[34]  Demetri Terzopoulos,et al.  Reconstructing and visualizing models of neuronal dendrites , 1991 .

[35]  C. D. Kuglin,et al.  The phase correlation image alignment method , 1975 .

[36]  George Wolberg,et al.  Digital image warping , 1990 .

[37]  Tomaso Poggio,et al.  Example Based Image Analysis and Synthesis , 1993 .

[38]  Michal Irani,et al.  Improving resolution by image registration , 1991, CVGIP Graph. Model. Image Process..

[39]  Dana H. Ballard,et al.  Computer Vision , 1982 .

[40]  Richard Szeliski,et al.  Recovering 3D Shape and Motion from Image Streams Using Nonlinear Least Squares , 1994, J. Vis. Commun. Image Represent..

[41]  Jake K. Aggarwal,et al.  Structure from stereo-a review , 1989, IEEE Trans. Syst. Man Cybern..

[42]  Michael Isard,et al.  3D position, attitude and shape input using video tracking of hands and lips , 1994, SIGGRAPH.

[43]  Richard Szeliski,et al.  Rapid octree construction from image sequences , 1993 .

[44]  Thomas S. Huang,et al.  Motion and Structure from Image Sequences , 1992 .

[45]  Richard Szeliski,et al.  Modeling and analysis of empirical data in collaborative environments , 1992, CACM.

[46]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[47]  Pierre David Wellner,et al.  Interacting with paper on the DigitalDesk , 1993, CACM.

[48]  Tom Duff,et al.  Compositing digital images , 1984, SIGGRAPH.

[49]  Richard Szeliski,et al.  Robust Shape Recovery from Occluding Contours Using a Linear Smoother , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[50]  Ned Greene,et al.  Environment Mapping and Other Applications of World Projections , 1986, IEEE Computer Graphics and Applications.

[51]  Alex Pentland,et al.  Closed-Form Solutions for Physically Based Shape Modeling and Recognition , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[52]  J. A. Adam,et al.  Virtual reality is for real , 1993 .

[53]  Takeo Kanade,et al.  Visual Tracking of High DOF Articulated Structures: an Application to Human Hand Tracking , 1994, ECCV.

[54]  Martin A. Fischler,et al.  Computational Stereo , 1982, CSUR.

[55]  Richard I. Hartley,et al.  Euclidean Reconstruction from Uncalibrated Views , 1993, Applications of Invariance in Computer Vision.

[56]  M. Rioux,et al.  White laser, synced scan , 1993 .

[57]  Verzekeren Naar Sparen,et al.  Cambridge , 1969, Humphrey Burton: In My Own Time.

[58]  J. G. Semple,et al.  Algebraic Projective Geometry , 1953 .