Haptics in augmented reality

An augmented reality system merges synthetic sensory information into a user's perception of a three-dimensional environment. An important performance goal for an augmented reality system is that the user perceives a single seamless environment. In most augmented reality systems the user views a real world augmented only with visual information and is not provided with a means to interact with the virtual objects. We describe an augmented reality system that, in addition to visual augmentation, merges synthetic haptic input into the user's perception of the real environment. Our system uses a PHANToM/sup TM/ haptic interface device to generate the haptic sensory input in real-time. The system allows user interactions such as moving or lifting a virtual object, and demonstrates interactions between virtual and real objects. Methods to provide proper visual occlusion between real and virtual objects are also described.

[1]  Rolf Ziegler,et al.  Haptic display for the VR arthroscopy training simulator , 1997, Electronic Imaging.

[2]  Frederick P. Brooks,et al.  Project GROPEHaptic displays for scientific visualization , 1990, SIGGRAPH.

[3]  T. P. Caudell,et al.  Augmented reality: an application of heads-up display technology to manual manufacturing processes , 1992, Proceedings of the Twenty-Fifth Hawaii International Conference on System Sciences.

[4]  Christian Breiteneder,et al.  TELEPORT—an augmented reality teleconferencing environment , 1996 .

[5]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[6]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[7]  Roger Mohr,et al.  Towards Fast and Realistic Image Synthesis from Real Views , 1997 .

[8]  Grigore C. Burdea,et al.  Virtual reality training simulation for palpation of subsurface tumors , 1997, Proceedings of IEEE 1997 Annual International Symposium on Virtual Reality.

[9]  O. Faugeras.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[10]  Matthias M. Wloka,et al.  Resolving occlusion in augmented reality , 1995, I3D '95.

[11]  Kiriakos N. Kutulakos,et al.  Calibration-Free Augmented Reality , 1998, IEEE Trans. Vis. Comput. Graph..

[12]  Jean-Luc Dugelay.  Synthetic natural hybrid video processings for virtual teleconferencing systems , 1999 .

[13]  Amnon Shashua,et al.  Robust recovery of camera rotation from three frames , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[14]  Mandayam A. Srinivasan,et al.  Phantom-Based Haptic Interaction with Virtual Objects , 1997, IEEE Computer Graphics and Applications.

[15]  Pierre David Wellner,et al.  Interacting with paper on the DigitalDesk , 1993, CACM.

[16]  Takeo Kanade,et al.  Virtual Space Teleconferencing Using a Sea of Cameras , 1994 .

[17]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Katia Fintzel.  Visual spatialization of meeting room from 2D uncalibrated views , 1998 .

[19]  Frederick P. Brooks,et al.  Moving objects in space: exploiting proprioception in virtual-environment interaction , 1997, SIGGRAPH.

[20]  Jean-Luc Dugelay.  Image reconstruction and interpolation in trinocular vision , 1996 .

[21]  Amnon Shashua,et al.  Trilinearity in Visual Recognition by Alignment , 1994, ECCV.

[22]  Yutaka Matsushita,et al.  Multiparty videoconferencing at virtual social distance: MAJIC design , 1994, CSCW '94.

[23]  Mark A. Livingston,et al.  Superior augmented reality registration by integrating landmark tracking and magnetic tracking , 1996, SIGGRAPH.

[24]  Mary C. Whitton,et al.  Technologies for augmented reality systems: realizing ultrasound-guided needle biopsies , 1996, SIGGRAPH.

[25]  Amnon Shashua,et al.  Projective Structure from Uncalibrated Images: Structure From Motion and Recognition , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Olivier D. Faugeras,et al.  What can two images tell us about a third one? , 1994, ECCV.

[27]  John C. Davis,et al.  Contouring: A Guide to the Analysis and Display of Spatial Data , 1992 .

[28]  David E. Breen,et al.  Calibration Requirements and Procedures for a Monitor-Based Augmented Reality System , 1995, IEEE Trans. Vis. Comput. Graph..

[29]  W. Eric L. Grimson,et al.  Evaluating and Validating an Automated Registration System for Enhanced Reality Visualization in Surgery , 1995, CVRMed.

[30]  Takeo Kanade,et al.  What you can see is what you can feel-development of a visual/haptic interface to virtual environment , 1996, Proceedings of the IEEE 1996 Virtual Reality Annual International Symposium.

[31]  Tomaso Poggio,et al.  Example Based Image Analysis and Synthesis , 1993 .