Multimodal Sensorimotor Integration for Expert-in-the-Loop Telerobotic Surgical Training

This paper presents a novel multimodal training platform integrated with hand-over-hand (HOH) haptic guidance for dual-console surgical robotic systems such as the da Vinci Si system. The expert-in-the-loop (EIL) framework incorporates a fuzzy interface system in order to provide a trainee with adaptive authority over the procedure as well as hand-over-hand haptic guidance adjusted in real time based on the proficiency level of the trainee. The EIL expertise-oriented framework enables performance of a surgical procedure by an expert surgeon on a patient, while simultaneously providing a trainee at any stage of the motor-skills development with multimodal training without jeopardizing patient safety. Closed-loop stability of the system is investigated using the circle criterion and it is shown that the proposed architecture is unconditionally stable. Experimental evaluations are presented in support of the proposed platform through the implementation of a dual-console surgical setup consisting of the classic da Vinci surgical system (Intuitive Surgical, Inc., Sunnyvale, CA, USA) and the dV-Trainer master console (Mimic Technology, Inc., Seattle, WA, USA). To the best of our knowledge, the implemented setup is the first research platform for dual-console studies involving the classic da Vinci surgical system.

[1]  Ilana Nisky,et al.  Robot-assisted surgery: an emerging platform for human neuroscience research , 2015, Front. Hum. Neurosci..

[2]  J. Kopta,et al.  The development of motor skills in orthopaedic education. , 1971, Clinical orthopaedics and related research.

[3]  Keyvan Hashtrudi-Zaad,et al.  A Framework for Unconditional Stability Analysis of Multimaster/Multislave Teleoperation Systems , 2013, IEEE Transactions on Robotics.

[4]  M. Kawato,et al.  Two is better than one: Physical interactions improve motor performance in humans , 2014, Scientific Reports.

[5]  Keyvan Hashtrudi-Zaad,et al.  Analysis of Coupled Stability in Multilateral Dual-User Teleoperation Systems , 2014, IEEE Transactions on Robotics.

[6]  Peter M. Vishton,et al.  Haptically Linked Dyads , 2006, Psychological science.

[7]  Stephane Cotin,et al.  Metrics for Laparoscopic Skills Trainers: The Weakest Link! , 2002, MICCAI.

[8]  Ali Talasaz,et al.  Haptics-Enabled Teleoperation for Robotics-Assisted Minimally Invasive Surgery , 2012 .

[9]  Tianjiao Chu,et al.  Dual-console robotic surgery: a new teaching paradigm , 2012, Journal of Robotic Surgery.

[10]  Ali Shahdi,et al.  Adaptive/Robust Control for Time-Delay Teleoperation , 2009, IEEE Transactions on Robotics.

[11]  T. Flash,et al.  Moving gracefully: quantitative theories of motor coordination , 1987, Trends in Neurosciences.

[12]  Martin Buss,et al.  Efficiency analysis in a collaborative task with reciprocal haptic feedback , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[13]  A. Okamura Haptic feedback in robot-assisted minimally invasive surgery , 2009, Current opinion in urology.

[14]  R. Reznick,et al.  Teaching surgical skills--changes in the wind. , 2006, The New England journal of medicine.

[15]  Ravishankar K. Iyer,et al.  Adverse Events in Robotic Surgery: A Retrospective Study of 14 Years of FDA Data , 2015, PloS one.

[16]  John W. Davis,et al.  Robotic surgery training with commercially available simulation systems in 2011: a current review and practice pattern survey from the society of urologic robotic surgeons. , 2012, Journal of endourology.

[17]  Peter Kazanzides,et al.  An open-source research kit for the da Vinci® Surgical System , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[18]  Martin Buss,et al.  Shared-Control Paradigms in Multi-Operator-Single-Robot Teleoperation , 2009, Human Centered Robot Systems, Cognition, Interaction, Technology.

[19]  Marcia Kilchenman O'Malley,et al.  Improving the retention of motor skills after reward-based reinforcement by incorporating haptic guidance and error augmentation , 2016, 2016 6th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[20]  M. S. Atkins,et al.  What Do Surgeons See , 2013, Surgical innovation.

[21]  Allison M. Okamura,et al.  Two is not always better than one: Effects of teleoperation and haptic coupling , 2016, 2016 6th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[22]  Septimiu E. Salcudean,et al.  A retrofit eye gaze tracker for the da Vinci and its integration in task execution using the da Vinci Research Kit , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[23]  Germain Forestier,et al.  Classification of surgical processes using dynamic time warping , 2012, J. Biomed. Informatics.

[24]  Peter Xiaoping Liu,et al.  A multichannel IOS small gain theorem for systems with multiple time-varying communication delays , 2007, 2007 46th IEEE Conference on Decision and Control.

[25]  Ilia G. Polushin,et al.  Cooperative Teleoperation With Projection-Based Force Reflection for MIS , 2015, IEEE Trans. Control. Syst. Technol..

[26]  Peter Kazanzides,et al.  A component-based architecture for flexible integration of robotic systems , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[27]  Mark L. Nagurka,et al.  Dynamic and loaded impedance components in the maintenance of human arm posture , 1993, IEEE Trans. Syst. Man Cybern..

[28]  Heidar Ali Talebi,et al.  An expertise-oriented training framework for robotics-assisted surgery , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[29]  Mahdi Tavakoli,et al.  Transparent Time-Delayed Bilateral Teleoperation Using Wave Variables , 2008, IEEE Transactions on Control Systems Technology.

[30]  Luka Peternel,et al.  Humanoid robot posture-control learning in real-time based on human sensorimotor learning ability , 2013, 2013 IEEE International Conference on Robotics and Automation.

[31]  C. Shea,et al.  Motor skill learning and performance: a review of influential factors , 2010, Medical education.

[32]  Mahdi Tavakoli,et al.  Absolute Stability of Multi-DOF Multilateral Haptic Systems , 2014, IEEE Transactions on Control Systems Technology.

[33]  Jerzy W. Rozenblit,et al.  SURGICAL TRAINING AND PERFORMANCE ASSESSMENT USING A MOTION TRACKING SYSTEM , 2006 .

[34]  Jerzy W. Rozenblit,et al.  Knowledge elicitation for performance assessment in a computerized surgical training system , 2011, Appl. Soft Comput..

[35]  Marcia Kilchenman O'Malley,et al.  Negative efficacy of fixed gain error reducing shared control for training in virtual environments , 2009, TAP.

[36]  Carsten W. Scherer,et al.  Stability Analysis for Bilateral Teleoperation: An IQC Formulation , 2012, IEEE Transactions on Robotics.