Weakly calibrated video-based augmented reality: embedding and rendering through virtual camera

Proposed is an algorithm for augmenting a real video sequence with views of graphics objects without metric calibration of the video camera by representing the motion of the video camera in projective space. We define a virtual camera, through which views of graphics objects are generated, attached to the real camera by specifying image locations of the world coordinate system of the virtual world. The virtual camera is decomposed into calibration and motion components in order to make full use of graphics tools. The projective motion of the real camera recovered from image matches has the function of transferring the virtual camera, and makes the virtual camera move according to the motion of the real camera. The virtual camera also follows the change of the internal parameters of the real camera. Real experiments show the performance of the algorithm.

[1]  Rajiv Gupta,et al.  Stereo from uncalibrated cameras , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  O. Faugeras,et al.  From geometry to variational calculus: theory and applications of three-dimensional vision , 1998, Proceedings 1998 IEEE and ATR Workshop on Computer Vision for Virtual Reality Based Human Communications.

[3]  Olivier D. Faugeras,et al.  What can be seen in three dimensions with an uncalibrated stereo rig , 1992, ECCV.

[4]  David E. Breen,et al.  Real-time vision-based camera tracking for augmented reality applications , 1997, VRST '97.

[5]  Didier Stricker,et al.  The Use of Reality Models in Augmented Reality Applications , 1998, SMILE.

[6]  Frank Siering,et al.  Applying Augmented Reality Techniques in the Field of Interactive Collaborative Design , 1998, SMILE.

[7]  Amnon Shashua,et al.  Projective depth: A geometric invariant for 3D reconstruction from two perspective/orthographic views and for visual recognition , 1993, 1993 (4th) International Conference on Computer Vision.

[8]  Seong-Woo Park,et al.  Real-Time Camera Calibration for Virtual Studio , 2000, Real Time Imaging.

[9]  David E. Breen,et al.  Automated Camera Calibration and 3D Egomotion Estimation for Augmented Reality Applications , 1997, CAIP.

[10]  Ulrich Neumann,et al.  Dynamic registration correction in video-based augmented reality systems , 1995, IEEE Computer Graphics and Applications.

[11]  Kiriakos N. Kutulakos,et al.  Calibration-Free Augmented Reality , 1998, IEEE Trans. Vis. Comput. Graph..

[12]  O. Faugeras.  Stratification of three-dimensional vision: projective, affine, and metric representations , 1995 .

[13]  Yongduek Seo,et al.  Calibration-Free Augmented Reality in Perspective , 2000, IEEE Trans. Vis. Comput. Graph..

[14]  Yi-Ping Hung,et al.  New calibration-free approach for augmented reality based on parameterized cuboid structure , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[15]  Marie-Odile Berger.  Resolving occlusion in augmented reality: a contour based approach without 3D reconstruction , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[17]  Olivier Faugeras,et al.  Three-Dimensional Computer Vision , 1993 .

[18]  Yongduek Seo,et al.  Video augmentation by image-based rendering under the perspective camera model , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).