Dynamic superimposition of synthetic objects on rigid and simple-deformable real objects

A current challenge in augmented reality applications is the accurate superimposition of synthetic objects on real objects within the environment. This challenge is heightened when the real objects are in motion and/or are nonrigid. In this article, we present a robust method for realtime, optical superimposition of synthetic objects on dynamic rigid and simple-deformable real objects. Moreover, we illustrate this general method with the VRDA Tool, a medical education application related to the visualization of internal human knee joint anatomy on a real human knee.

[1]  Larry S. Davis,et al.  Technologies for Augmented Reality: Calibration for Real-Time Superimposition on Rigid and Simple-Deformable Real Objects , 2001, MICCAI.

[2]  Eric Foxlin,et al.  Inertial head-tracker sensor fusion by a complementary separate-bias Kalman filter , 1996, Proceedings of the IEEE 1996 Virtual Reality Annual International Symposium.

[3]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[4]  G. Stewart.  Introduction to matrix computations , 1973 .

[5]  Ramesh Raskar,et al.  Augmented Reality Visualization for Laparoscopic Surgery , 1998, MICCAI.

[6]  Kuo-Chi Lin,et al.  Automatic Modeling of Knee-Joint Motion For The Virtual Reality Dynamic Anatomy (VRDA) Tool , 2000, Presence: Teleoperators & Virtual Environments.

[7]  K. S. Arun,et al.  Least-Squares Fitting of Two 3-D Point Sets , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Larry F. Hodges,et al.  Generating exact ray-traced animation frames by reprojection , 1995, IEEE Computer Graphics and Applications.

[9]  Ronald Azuma,et al.  Improving static and dynamic registration in an optical see-through HMD , 1994, SIGGRAPH.

[10]  J. Rolland,et al.  Using virtual reality to teach radiographic positioning. , 1995, Radiologic technology.

[11]  T P Andriacchi,et al.  A point cluster method for in vivo motion analysis: applied to a study of knee kinematics. , 1998, Journal of biomechanical engineering.

[12]  Alberto Leardini,et al.  Skin movement artefacts in human movement photogrammetry , 1994 .

[13]  Greg Welch,et al.  SCAAT: incremental tracking with incomplete information , 1997, SIGGRAPH.

[14]  W. Press,et al.  Numerical Recipes in C++: The Art of Scientific Computing (2nd edn)1 Numerical Recipes Example Book (C++) (2nd edn)2 Numerical Recipes Multi-Language Code CD ROM with LINUX or UNIX Single-Screen License Revised Version3 , 2003 .

[15]  C. Spoor,et al.  Rigid body motion calculated from spatial co-ordinates of markers. , 1980, Journal of biomechanics.

[16]  Larry S. Davis,et al.  Dynamic superimposition of synthetic objects on rigid and simple-deformable real objects , 2002, Comput. Graph..

[17]  Jennifer Healey,et al.  Augmented Reality through Wearable Computing , 1997, Presence: Teleoperators & Virtual Environments.

[18]  Jannick P. Rolland,et al.  Method of Adjustments versus Method of Constant Stimuli in the Quantification of Accuracy and Precision of Rendered Depth in Head-Mounted Displays , 2002, Presence: Teleoperators & Virtual Environments.

[19]  Henry Fuchs,et al.  Optical Versus Video See-Through Head-Mounted Displays in Medical Visualization , 2000, Presence: Teleoperators & Virtual Environments.

[20]  J. P. Mellor.  Realtime Camera Calibration for Enhanced Reality , 1995 .

[21]  Ivan E. Sutherland.  A head-mounted three dimensional display , 1998 .

[22]  Dorin Comaniciu,et al.  Real-time tracking of non-rigid objects using mean shift , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[23]  Parravano Jg,et al.  Postprocessing conventional vs. fast spin echo MR images. , 1995 .

[24]  V. K. Utyonkov,et al.  Voyage to Superheavy Island. , 2000 .

[25]  Mark A. Livingston,et al.  Superior augmented reality registration by integrating landmark tracking and magnetic tracking , 1996, SIGGRAPH.

[26]  Daphna Weinshall,et al.  Motion of disturbances: detection and tracking of multi-body non-rigid motion , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[27]  Ivan E. Sutherland,et al.  A head-mounted three dimensional display , 1968, AFIPS Fall Joint Computing Conference.

[28]  Ulrich Neumann,et al.  Dynamic registration correction in video-based augmented reality systems , 1995, IEEE Computer Graphics and Applications.

[29]  Jannick P. Rolland,et al.  Towards Quantifying Depth and Size Perception in Virtual Environments , 1993, Presence: Teleoperators & Virtual Environments.

[30]  .. McGlone,et al.  Vision-Based Object Registration for Real-Time Image Overlay , 1995 .

[31]  Steven K. Feiner,et al.  Knowledge-based augmented reality , 1993, CACM.

[32]  Warren Robinett,et al.  A Computational Model for the Stereoscopic Optics of a Head-Mounted Display , 1991, Presence: Teleoperators & Virtual Environments.

[33]  T. P. Caudell,et al.  Augmented reality: an application of heads-up display technology to manual manufacturing processes , 1992, Proceedings of the Twenty-Fifth Hawaii International Conference on System Sciences.

[34]  J P Rolland,et al.  Engineering of head-mounted projective displays. , 2000, Applied optics.

[35]  Marcel Rooze,et al.  Visualization of Combined Motions in Human Joints , 1998, IEEE Computer Graphics and Applications.

[36]  U.D. Croce,et al.  Surface-marker cluster design criteria for 3-D bone movement reconstruction , 1997, IEEE Transactions on Biomedical Engineering.

[37]  Hirokazu Kato,et al.  Collaborative Mixed Reality , 1999 .

[38]  R. Kikinis,et al.  Image-guided surgery. , 1999, Scientific American.