Multi-modal Virtual Reality Dental Training System with Integrated Haptic-Visual-Audio Display

Multi-modal signal fusion is an important way to improve immersion and fidelity of virtual reality training systems. With the help of a robotic arm, haptic device, it is possible to supply haptic-visual-audio display for training dentist. Architecture is firstly proposed to integrate different signals in a training system. High efficient sound synthesis algorithm is proposed based on hybrid combination of parameter identification from recorded sound signal, then use signal synthesis method to form final sound signal. Mapping method among audio, haptic and visual signal is proposed based on parameter mapping under various interaction status. Experiment based on Phantom desktop proves the stability of the rendering algorithm and the improved fidelity after adding audio feedback to current haptic-visual dental training system

[1]  Yuan-Shin Lee,et al.  Cutting on triangle mesh: local model-based haptic display for dental preparation surgery simulation , 2005, IEEE Transactions on Visualization and Computer Graphics.

[2]  Lonny L. Chu.  User performance and haptic design issues for a force-feedback sound editing interface , 2002, CHI Extended Abstracts.

[3]  R. Riener,et al.  Phantom-based multimodal interactions for medical education and training: the Munich knee joint simulator , 2004, IEEE Transactions on Information Technology in Biomedicine.

[4]  Martin Buss,et al.  Distributed PC-based haptic, visual and acoustic telepresence system-experiments in virtual and remote environments , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[5]  Hiroaki Yano,et al.  AudioHaptics: audio and haptic rendering based on a physical model , 2004, 12th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2004. HAPTICS '04. Proceedings..

[6]  M. Bergamasco,et al.  Computer graphics access for blind people through a haptic and audio virtual environment , 2004, Proceedings. Second International Conference on Creating, Connecting and Collaborating through Computing.

[7]  Dinesh K. Pai,et al.  The Sounds of Physical Shapes , 1998, Presence.