Altering reality through interactive image and video manipulation

Camera calibration and the acquisition of Euclidean 3D measurements have so far been considered necessary requirements for overlaying three-dimensional graphical objects with live video. We describe an image-based approach to augmented reality that avoids both requirements: it does not use any metric information about the calibration parameters of the camera or the 3D locations and dimensions of the environment's objects. The only requirement is the ability to track across frames at least four feature points that are specified by the user during system initialization and whose world coordinates are unknown. The resulting system allows a user to "augment" a real 3D environment with computer-generated objects by interactively manipulating one or more images of that environment.

[1]  .. McGlone,et al.  Vision-Based Object Registration for Real-Time Image Overlay , 1995 .

[2]  D. W. Thompson,et al.  Three-dimensional model matching from an unconstrained viewpoint , 1987, Proceedings. 1987 IEEE International Conference on Robotics and Automation.

[3]  Alex Pentland,et al.  A novel environment for situated vision and behavior , 1994 .

[4]  O. Faugeras.  Stratification of three-dimensional vision: projective, affine, and metric representations , 1995 .

[5]  Mark A. Livingston,et al.  Superior augmented reality registration by integrating landmark tracking and magnetic tracking , 1996, SIGGRAPH.

[6]  Bruce A. Draper,et al.  Adaptive tracking and model registration across distinct aspects , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[7]  M. Carter.  Computer graphics: Principles and practice , 1997 .

[8]  Andrew Zisserman,et al.  Geometric invariance in computer vision , 1992 .

[9]  Ronen Basri,et al.  Recognition by Linear Combinations of Models , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Kiriakos N. Kutulakos,et al.  Affine object representations for calibration-free augmented reality , 1996, Proceedings of the IEEE 1996 Virtual Reality Annual International Symposium.

[11]  Matthias M. Wloka,et al.  Resolving occlusion in augmented reality , 1995, I3D '95.

[12]  Roberto Cipolla,et al.  Uncalibrated Stereo Vision with Pointing for a Man-Machine Interface , 1994, MVA.

[13]  Richard L. Holloway,et al.  Registration errors in augmented reality systems , 1996 .

[14]  O. Faugeras.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[15]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[16]  Mary C. Whitton,et al.  Technologies for augmented reality systems: realizing ultrasound-guided needle biopsies , 1996, SIGGRAPH.

[17]  J. P. Mellor,et al.  Enhanced Reality Visualization in a Surgical Environment , 1995 .

[18]  Ulrich Neumann,et al.  Dynamic registration correction in video-based augmented reality systems , 1995, IEEE Computer Graphics and Applications.

[19]  Ryutarou Ohbuchi,et al.  Merging virtual objects with the real world: seeing ultrasound imagery within the patient , 1992, SIGGRAPH.

[20]  J J Koenderink,et al.  Affine structure from motion. , 1991, Journal of the Optical Society of America. A, Optics and image science.

[21]  David E. Breen,et al.  Calibration Requirements and Procedures for a Monitor-Based Augmented Reality System , 1995, IEEE Trans. Vis. Comput. Graph..

[22]  W. Eric L. Grimson,et al.  Evaluating and Validating an Automated Registration System for Enhanced Reality Visualization in Surgery , 1995, CVRMed.

[23]  Michael Gleicher,et al.  Through-the-lens camera control , 1992, SIGGRAPH.

[24]  Kiriakos N. Kutulakos,et al.  Calibration-Free Augmented Reality , 1998, IEEE Trans. Vis. Comput. Graph..

[25]  T. P. Caudell,et al.  Augmented reality: an application of heads-up display technology to manual manufacturing processes , 1992, Proceedings of the Twenty-Fifth Hawaii International Conference on System Sciences.

[26]  Steven K. Feiner,et al.  Knowledge-based augmented reality , 1993, CACM.