Skill-based human–robot cooperation in tele-operated path tracking

This work proposes a shared-control tele-operation framework that adapts its cooperative properties to the estimated skill level of the operator. It is hypothesized that different aspects of an operator’s performance in executing a tele-operated path tracking task can be assessed through conventional machine learning methods using motion-based and task-related features. To identify performance measures that capture motor skills linked to the studied task, an experiment is conducted where users new to tele-operation, practice towards motor skill proficiency in 7 training sessions. A set of classifiers are then learned from the acquired data and selected features, which can generate a skill profile that comprises estimations of user’s various competences. Skill profiles are exploited to modify the behavior of the assistive robotic system accordingly with the objective of enhancing user experience by preventing unnecessary restriction for skilled users. A second experiment is implemented in which novice and expert users execute the path tracking on different pathways while being assisted by the robot according to their estimated skill profiles. Results validate the skill estimation method and hint at feasibility of shared-control customization in tele-operated path tracking.

[1]  Su-Lin Lee,et al.  Learning-Based Modeling of Endovascular Navigation for Collaborative Robotic Catheterization , 2013, MICCAI.

[2]  Luis F. Peñín,et al.  Force reflection for time-delayed teleoperation of Space robots , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[3]  Nima Enayati,et al.  Design and manufacturing of a tele-operative rescue robot with a novel track arrangement , 2011, Ind. Robot.

[4]  Ratna Babu Chinnam,et al.  Soft Boundary Approach for Unsupervised Gesture Segmentation in Robotic-Assisted Surgery , 2017, IEEE Robotics and Automation Letters.

[5]  Gregory D. Hager,et al.  Towards automatic skill evaluation: detection and segmentation of robot-assisted surgical motions. , 2006 .

[6]  M. Schijven,et al.  Training and learning robotic surgery, time for a more structured approach: a systematic review , 2012, BJOG : an international journal of obstetrics and gynaecology.

[7]  Gregory D. Hager,et al.  Task versus Subtask Surgical Skill Evaluation of Robotic Minimally Invasive Surgery , 2009, MICCAI.

[8]  A. Knoll,et al.  Toward increased autonomy in the surgical OR: needs, requests, and expectations , 2013, Surgical Endoscopy.

[9]  Thomas Schamm,et al.  Contextual task-aware shared autonomy for assistive mobile robot teleoperation , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  G Ferrigno,et al.  Enhanced torque‐based impedance control to assist brain targeting during open‐skull neurosurgery: a feasibility study , 2016, The international journal of medical robotics + computer assisted surgery : MRCAS.

[11]  Leila Takayama,et al.  Assisted driving of a mobile remote presence system: System design and controlled user evaluation , 2011, 2011 IEEE International Conference on Robotics and Automation.

[12]  Richard A. Schmidt,et al.  Frequent Augmented Feedback Can Degrade Learning: Evidence and Interpretations , 1991 .

[13]  E. Verdaasdonk,et al.  Objective assessment of technical surgical skills , 2010, The British journal of surgery.

[14]  Allison M. Okamura,et al.  Models of human-centered automation in a debridement task , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[15]  Allison M. Okamura,et al.  Haptic Virtual Fixtures for Robot-Assisted Manipulation , 2005, ISRR.

[16]  Nima Enayati,et al.  A dynamic non-energy-storing guidance constraint with motion redirection for robot-assisted surgery , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[17]  Guang-Zhong Yang,et al.  Hubot: A three state Human-Robot collaborative framework for bimanual surgical tasks based on learned models , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[18]  Ferdinando Rodriguez y Baena,et al.  Dissipative Control for Physical Human–Robot Interaction , 2015, IEEE Transactions on Robotics.

[19]  Paula Gomes,et al.  Surgical robotics: Reviewing the past, analysing the present, imagining the future , 2011 .

[20]  Herman Bruyninckx,et al.  Open robot control software: the OROCOS project , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[21]  Blake Hannaford,et al.  Generalized approach for modeling minimally invasive surgery as a stochastic process using a discrete Markov model , 2006, IEEE Transactions on Biomedical Engineering.

[22]  Antonio Franchi,et al.  Shared Control : Balancing Autonomy and Human Assistance with a Group of Quadrotor UAVs , 2012, IEEE Robotics & Automation Magazine.

[23]  J. Dankelman,et al.  Objective classification of residents based on their psychomotor laparoscopic skills , 2009, Surgical Endoscopy.

[24]  Sterling J. Anderson,et al.  Constraint-based planning and control for safe, semi-autonomous operation of vehicles , 2012, 2012 IEEE Intelligent Vehicles Symposium.

[25]  Nima Enayati,et al.  Performance metrics for guidance active constraints in surgical robotics , 2018, The international journal of medical robotics + computer assisted surgery : MRCAS.

[26]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[27]  Francisco José Madrid-Cuevas,et al.  Automatic generation and detection of highly reliable fiducial markers under occlusion , 2014, Pattern Recognit..

[28]  K. J. Kuchenbecker,et al.  Lack of a discriminatory function for endoscopy skills on a computer-based simulator , 2010, Surgical Endoscopy.

[29]  Shahram Payandeh,et al.  Fuzzy Set Theory for Performance Evaluation in a Surgical Simulator , 2007, PRESENCE: Teleoperators and Virtual Environments.

[30]  Rajnikant V. Patel,et al.  The application of force sensing to skills assessment in Minimally Invasive Surgery , 2013, 2013 IEEE International Conference on Robotics and Automation.

[31]  John W. Krakauer,et al.  Independent learning of internal models for kinematic and dynamic control of reaching , 1999, Nature Neuroscience.

[32]  Nima Enayati,et al.  Haptics in Robot-Assisted Surgery: Challenges and Benefits , 2016, IEEE Reviews in Biomedical Engineering.

[33]  M. Finan,et al.  A novel method for training residents in robotic hysterectomy , 2010, Journal of robotic surgery.

[34]  Paolo Dario,et al.  Modelling and Evaluation of Surgical Performance Using Hidden Markov Models , 2006, IEEE Transactions on Biomedical Engineering.

[35]  R. Sainburg,et al.  Interlimb transfer of visuomotor rotations: independence of direction and final position information , 2002, Experimental Brain Research.

[36]  Siddhartha S. Srinivasa,et al.  A policy-blending formalism for shared control , 2013, Int. J. Robotics Res..

[37]  K. Newell,et al.  Qualitative and quantitative change in the dynamics of motor learning. , 2006, Journal of experimental psychology. Human perception and performance.

[38]  J. Krakauer,et al.  Error correction, sensory prediction, and adaptation in motor control. , 2010, Annual review of neuroscience.

[39]  Stephane Cotin,et al.  Metrics for Laparoscopic Skills Trainers: The Weakest Link! , 2002, MICCAI.

[40]  Gregory D. Hager,et al.  A Dataset and Benchmarks for Segmentation and Recognition of Gestures in Robotic Surgery , 2017, IEEE Transactions on Biomedical Engineering.

[41]  Victor F. Muñoz,et al.  Collaborative Human–Robot System for HALS Suture Procedures , 2016, IEEE Systems Journal.

[42]  B. Hannaford,et al.  Skills evaluation in minimally invasive surgery using force/torque signatures , 2000, Surgical Endoscopy.

[43]  Brian L. Davies,et al.  Active Constraints/Virtual Fixtures: A Survey , 2014, IEEE Transactions on Robotics.

[44]  Dale A. Lawrence.  Stability and transparency in bilateral teleoperation , 1993, IEEE Trans. Robotics Autom..

[45]  David J. Reinkensmeyer,et al.  Haptic Guidance Can Enhance Motor Learning of a Steering Task , 2008, Journal of motor behavior.

[46]  Gregory D. Hager,et al.  Human-Machine Collaborative surgery using learned models , 2011, 2011 IEEE International Conference on Robotics and Automation.

[47]  J. Krakauer,et al.  How is a motor skill learned? Change and invariance at the levels of task success and trajectory control. , 2012, Journal of neurophysiology.

[48]  R. Riener,et al.  Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review , 2012, Psychonomic Bulletin & Review.