On the role of multimodal communication in telesurgery systems

Telesurgery systems integrate multimodal communication and robotic technologies to enable surgical procedures to be performed from remote locations. They allow human surgeons to intuitively control laparoscopic instruments and to navigate within the human body. In this paper, we present selected topics on multimodal interaction in the context of telesurgery applications. These are results from the collaborative research project SFB 453 on “High-Fidelity Telepresence and Teleaction” which is funded by the German Research Foundation in the larger Munich area. The focus in this paper is on multimodal information processing and communication including simulation of surgical targets in the human body. Furthermore, we present an overview of our advanced multimodal telesurgery demonstrators that provide a comprehensive platform for our collaborative telepresence research.

[1]  Bernhard Kübler,et al.  Sensorized and Actuated Instruments for Minimally Invasive Robotic Surgery , 2004 .

[2]  Sandra Hirche,et al.  Transparent Data Reduction in Networked Telepresence and Teleaction Systems. Part I: Communication without Time Delay , 2007, PRESENCE: Teleoperators and Virtual Environments.

[3]  Chen Zhao,et al.  Force modeling of inhomogeneous material using unsupervised learning and model identification , 2009, 2008 IEEE International Conference on Robotics and Biomimetics.

[4]  Leonard McMillan,et al.  Stable real-time deformations , 2002, SCA '02.

[5]  H. Rothert,et al.  Formulation and implementation of three-dimensional viscoelasticity at small and finite strains , 1997 .

[6]  Gerd Hirzinger,et al.  DLR MiroSurge -- towards versatility in surgical robotics , 2008 .

[7]  P. Hinterseer,et al.  Model based data compression for 3D virtual haptic teleinteraction , 2006, 2006 Digest of Technical Papers International Conference on Consumer Electronics.

[8]  Sandra Hirche,et al.  Perception-Based Data Reduction and Transmission of Haptic Data in Telepresence and Teleaction Systems , 2008, IEEE Transactions on Signal Processing.

[9]  Jie Ren,et al.  Investigations into Performance of Minimally Invasive Telesurgery with Feedback Time Delays , 2000, Presence: Teleoperators & Virtual Environments.

[10]  Bernard D. Adelstein,et al.  Head Tracking Latency in Virtual Environments: Psychophysics and a Model , 2003 .

[11]  Alin Albu-Schäffer,et al.  The DLR MIRO: a versatile lightweight robot for surgical applications , 2008, Ind. Robot.

[12]  Hermann J. Müller,et al.  Influence of visuomotor action on visual-haptic simultaneous perception: A psychophysical study , 2008, 2008 Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[13]  Sandra Hirche,et al.  Transparent Data Reduction in Networked Telepresence and Teleaction Systems. Part II: Time-Delayed Communication , 2007, PRESENCE: Teleoperators and Virtual Environments.

[14]  Stephane Cotin,et al.  EP4A: Software and Computer Based Simulator Research: Development and Outlook SOFA—An Open Source Framework for Medical Simulation , 2007, MMVR.

[15]  J. Marescaux,et al.  Transcontinental Robot-Assisted Remote Telesurgery: Feasibility and Potential Applications , 2002, Annals of surgery.

[16]  Caroline Jay,et al.  Modeling the effects of delayed haptic and visual feedback in a collaborative virtual environment , 2007, TCHI.

[17]  Mamoru Mitsuishi,et al.  Tele-micro-surgery system with intelligent user interface , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[18]  Vincent Hayward,et al.  Haptic Simulation of a Tool in Contact with a Nonlinear Deformable Body , 2003, IS4TH.

[19]  Sandra Hirche,et al.  Bilateral Control Architectures for Telerobotics , 2007, Advances in Telerobotics.

[20]  Mark W. Spong,et al.  Bilateral teleoperation: An historical survey , 2006, Autom..

[21]  Michael F. Zäh,et al.  Prediction-based methods for teleoperation across delayed networks , 2007, Multimedia Systems.

[22]  Sandra Hirche,et al.  A novel, psychophysically motivated transmission approach for haptic data streams in telepresence and teleaction systems , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[23]  M. Young,et al.  Discrimination of Changes of Latency during Voluntary Hand Movement of Virtual Objects , 1999 .