Augmenting panoramas with object movies by generating novel views with disparity-based view morphing

Our goal is to augment a panorama with object movies in a visually 3D-consistent way. Notice that a panorama is recorded as one single 2D image and an object movie (OM) is composed of a set of 2D images taken around a 3D object. The challenge is how to integrate the above two sources of 2D images in a 3D-consistent way so that the user can easily manipulate object movies in a panorama. To solve this problem, we adopt a purely image-based approach that does not have to reconstruct the geometric models of the 3D objects to be inserted in the panorama. A critical issue of this method is how to generate the novel views required for showing an OM in different places of a panorama, and we have proposed a view morphing technique, called t-DBVM, to solve this problem. Our experiments have shown that this purely image-based approach can effectively generate visually convincing OM-augmented panoramas. This method has great potential for many applications that require integration of panoramas and object movies, such as virtual malls, virtual museum, and interior design. Copyright © 2002 John Wiley & Sons, Ltd.

[1]  Yi-Ping Hung,et al.  New calibration-free approach for augmented reality based on parameterized cuboid structure , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[2]  Kiriakos N. Kutulakos,et al.  A Theory of Shape by Space Carving , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[3]  Chu-Song Chen,et al.  Composition of 3D Graphic Objects and Panoramas , 2000 .

[4]  Yi-Ping Hung,et al.  Disparity-based view morphing—a new technique for image-based rendering , 1998, VRST '98.

[5]  Richard Szeliski,et al.  Creating full view panoramic image mosaics and texture-mapped models , 1997, International Conference on Computer Graphics and Interactive Techniques.

[6]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[7]  Chu-Song Chen,et al.  Composition of 3 D Graphic Objects and Panorama , 2000 .

[8]  Steven M. Seitz,et al.  View morphing , 1996, SIGGRAPH.

[9]  Hideyuki Tamura,et al.  Presentation and Interaction of Virtual 3D Objects Without Geometrical Model , 1997, HCI.

[10]  Yukio Sakagawa,et al.  Shading and shadow casting in image-based rendering without geometric models , 1999, SIGGRAPH '99.

[11]  Emanuele Trucco,et al.  Geometric Invariance in Computer Vision , 1995 .

[12]  Yongduek Seo,et al.  Calibration-Free Augmented Reality in Perspective , 2000, IEEE Trans. Vis. Comput. Graph..

[13]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[14]  Yi-Ping Hung,et al.  Panoramic Stereo Imaging System with Automatic Disparity Warping and Seaming , 1998, Graph. Model. Image Process..

[15]  Shree K. Nayar,et al.  Ego-motion and omnidirectional cameras , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[16]  Kiriakos N. Kutulakos,et al.  Calibration-Free Augmented Reality , 1998, IEEE Trans. Vis. Comput. Graph..

[17]  David E. Breen,et al.  Real-time vision-based camera tracking for augmented reality applications , 1997, VRST '97.

[18]  Chih-Han Yu,et al.  A Purely Image-Based Approach to Augmenting Panoramas with Object Movies , 2001 .

[19]  Paul E. Debevec,et al.  Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography , 1998, SIGGRAPH '08.

[20]  Yi-Ping Hung,et al.  Disparity-based view interpolation for multiple-perspective stereoscopic displays , 2000, Electronic Imaging.

[21]  Frank Siering,et al.  Applying Augmented Reality Techniques in the Field of Interactive Collaborative Design , 1998, SMILE.

[22]  Reinhard Koch,et al.  3D Structure from Multiple Images of Large-Scale Environments , 1998, Lecture Notes in Computer Science.

[23]  Shenchang Eric Chen,et al.  QuickTime VR: an image-based approach to virtual environment navigation , 1995, SIGGRAPH.