Non-Technical Skill Assessment and Mental Load Evaluation in Robot-Assisted Minimally Invasive Surgery

BACKGROUND: Sensor technologies and data collection practices are changing and improving quality metrics across various domains. Surgical skill assessment in Robot-Assisted Minimally Invasive Surgery (RAMIS) is essential for training and quality assurance. The mental workload on the surgeon (such as time criticality, task complexity, distractions) and non-technical surgical skills (including situational awareness, decision making, stress resilience, communication, leadership) may directly influence the clinical outcome of the surgery. METHODS: A literature search in PubMed, Scopus and PsycNet databases was conducted for relevant scientific publications. The standard PRISMA method was followed to filter the search results, including non-technical skill assessment and mental/cognitive load and workload estimation in RAMIS. Publications related to traditional manual Minimally Invasive Surgery were excluded, and also the usability studies on the surgical tools were not assessed. RESULTS: 50 relevant publications were identified for non-technical skill assessment and mental load and workload estimation in the domain of RAMIS. The identified assessment techniques ranged from self-rating questionnaires and expert ratings to autonomous techniques, citing their most important benefits and disadvantages. CONCLUSIONS: Despite the systematic research, only a limited number of articles was found, indicating that non-technical skill and mental load assessment in RAMIS is not a well-studied area. Workload assessment and soft skill measurement do not constitute part of the regular clinical training and practice yet. Meanwhile, the importance of the research domain is clear based on the publicly available surgical error statistics. Questionnaires and expert-rating techniques are widely employed in traditional surgical skill assessment; nevertheless, recent technological development in sensors and Internet of Things-type devices show that skill assessment approaches in RAMIS can be much more profound employing automated solutions. Measurements and especially big data type analysis may introduce more objectivity and transparency to this critical domain as well. SIGNIFICANCE: Non-technical skill assessment and mental load evaluation in Robot-Assisted Minimally Invasive Surgery is not a well-studied area yet; while the importance of this domain from the clinical outcome’s point of view is clearly indicated by the available surgical error statistics.

[1]  J. Ruiz-Rabelo,et al.  Validation of the NASA-TLX Score in Ongoing Assessment of Mental Workload During a Laparoscopic Learning Curve in Bariatric Surgery , 2015, Obesity Surgery.

[2]  S. Bunce,et al.  Functional near-infrared spectroscopy , 2006, IEEE Engineering in Medicine and Biology Magazine.

[3]  Masakatsu G. Fujie,et al.  Operability evaluation using an simulation system for gripping motion in robotic tele-surgery , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[4]  Somayeh B. Shafiei,et al.  Understanding Cognitive Performance During Robot-Assisted Surgery. , 2015, Urology.

[5]  Cindy H. Lio,et al.  A Mental Workload Study on the 2d and 3d Viewing Conditions of the da Vinci Surgical Robot , 2009 .

[6]  D. Eisenstein,et al.  Quality of Communication in Robotic Surgery and Surgical Outcomes , 2016, JSLS : Journal of the Society of Laparoendoscopic Surgeons.

[7]  H. G. van der Poel,et al.  Cognitive training for technical and non‐technical skills in robotic surgery: a randomised controlled trial , 2018, BJU international.

[8]  Adolfo Peña,et al.  The Dreyfus model of clinical problem-solving skills acquisition: a critical perspective , 2010, Medical education online.

[9]  Thenkurussi Kesavadas,et al.  Augmented‐reality‐based skills training for robot‐assisted urethrovesical anastomosis: a multi‐institutional randomised controlled trial , 2015, BJU international.

[10]  R. Flin,et al.  Non-technical skills for surgeons in the operating room: a review of the literature. , 2006, Surgery.

[11]  Guang-Zhong Yang,et al.  Cognitive Burden Estimation for Visuomotor Learning with fNIRS , 2010, MICCAI.

[12]  K. Cleary,et al.  State of the Art in Surgical Robotics: Clinical Applications and Technology Challenges , 2001 .

[13]  Michael B. McCamy,et al.  Saccadic Eye Movement Metrics Reflect Surgical Residents' Fatigue , 2014, Annals of surgery.

[14]  K. Guru,et al.  Improving Teamwork: Evaluating Workload of Surgical Team During Robot-assisted Surgery. , 2017, Urology.

[15]  Torsten B Neilands,et al.  The Safety Attitudes Questionnaire: psychometric properties, benchmarking data, and emerging research , 2006, BMC Health Services Research.

[16]  P. Chiu,et al.  Randomized controlled trial of EndoWrist-enabled robotic versus human laparoendoscopic single-site access surgery (LESS) in the porcine model , 2018, Surgical Endoscopy.

[17]  Prokar Dasgupta,et al.  Defining and Validating Non-technical Skills Training in Robotics , 2021 .

[18]  Manreet Kaur,et al.  Wisconsin Card Sorting Test: Normative data and experience , 2006, Indian journal of psychiatry.

[19]  Sarah Feldt Muldoon,et al.  Functional Brain States Measure Mentor-Trainee Trust during Robot-Assisted Surgery , 2018, Scientific Reports.

[20]  Scientific Skill Assessment of Basic Surgical Dissection and Overall Laparoscopic Performance. , 2017, Journal of endourology.

[21]  P. Dasgupta,et al.  Development and validation of a tool for non-technical skills evaluation in robotic surgery—the ICARS system , 2017, Surgical Endoscopy.

[22]  S. Thangaratinam,et al.  The Delphi technique , 2005 .

[23]  Face, content, construct, and concurrent validity of a novel robotic surgery patient-side simulator: the Xperience™ Team Trainer , 2016, Surgical Endoscopy.

[24]  I. Broeders,et al.  Ergonomics, user comfort, and performance in standard and robot-assisted laparoscopic surgery , 2008, Surgical Endoscopy.

[25]  C. Walters,et al.  Maximizing Efficiency and Reducing Robotic Surgery Costs Using the NASA Task Load Index , 2017, AORN journal.

[26]  Gerald Matthews,et al.  Mental workload and stress perceived by novice operators in the laparoscopic and robotic minimally invasive surgical interfaces. , 2012, Journal of endourology.

[27]  Ehsanollah Habibi,et al.  Evaluation of Rating Scale Mental Effort (RSME) effectiveness for mental workload assessment in nurses , 2016 .

[28]  Christine L. Lisetti,et al.  Using Noninvasive Wearable Computers to Recognize Human Emotions from Physiological Signals , 2004, EURASIP J. Adv. Signal Process..

[29]  Xiaoyang Jin,et al.  Adapting the short form of the Coping Inventory for Stressful Situations into Chinese , 2017, Neuropsychiatric disease and treatment.

[30]  Gerald Matthews,et al.  Multidimensional Profiling of Task Stress States for Human Factors , 2016, Hum. Factors.

[31]  Mark R. Wilson,et al.  Development and Validation of a Surgical Workload Measure: The Surgery Task Load Index (SURG-TLX) , 2011, World Journal of Surgery.

[32]  Sarah Henrickson Parker,et al.  Is the "sterile cockpit" concept applicable to cardiovascular surgery critical intervals or critical events? The impact of protocol-driven communication during cardiopulmonary bypass. , 2010, The Journal of thoracic and cardiovascular surgery.

[33]  An Ergonomic Assessment Of Four Different Donor Nephrectomy Approaches For The Surgeons And Their Assistants , 2019, Research and reports in urology.

[34]  I. Pavlidis,et al.  Fast by Nature - How Stress Patterns Define Human Experience and Performance in Dexterous Tasks , 2012, Scientific Reports.

[35]  Tamas Haidegger,et al.  Autonomy for Surgical Robots: Concepts and Paradigms , 2019, IEEE Transactions on Medical Robotics and Bionics.

[36]  Nassib G. Chamoun,et al.  An introduction to bispectral analysis for the electroencephalogram , 1994, Journal of Clinical Monitoring.

[37]  Keno März,et al.  Toward a standard ontology of surgical process models , 2018, International Journal of Computer Assisted Radiology and Surgery.

[38]  K. Chinzei.  Safety of Surgical Robots and IEC 80601-2-77: The First International Standard for Surgical Robots , 2019, Acta Polytechnica Hungarica.

[39]  Peter Kazanzides,et al.  Mobile Teleoperation: Evaluation of Wireless Wearable Sensing of the Operator's Arm Motion , 2021, ArXiv.

[40]  K. Guru,et al.  Do surgeon non-technical skills correlate with teamwork-related outcomes during robot-assisted surgery? , 2019, BMJ Leader.

[41]  Masakatsu G. Fujie,et al.  Development of Real-Time Simulation for Workload Quantization in Robotic Tele-surgery , 2006, 2006 IEEE International Conference on Robotics and Biomimetics.

[42]  Paula Gomes,et al.  Surgical robotics: Reviewing the past, analysing the present, imagining the future , 2011 .

[43]  Nick Sevdalis,et al.  The impact of nontechnical skills on technical performance in surgery: a systematic review. , 2012, Journal of the American College of Surgeons.

[44]  Nigel H. Lovell,et al.  Advanced Intelligent Systems for Surgical Robotics , 2020, Adv. Intell. Syst..

[45]  Mark R. Wilson,et al.  Surgeons’ display reduced mental effort and workload while performing robotically assisted surgical tasks, when compared to conventional laparoscopy , 2015, Surgical Endoscopy.

[46]  Demetrios Demetriades,et al.  The impact of heat stress on operative performance and cognitive function during simulated laparoscopic operative tasks. , 2015, Surgery.

[47]  Gyusung I. Lee,et al.  Comparative assessment of physical and cognitive ergonomics associated with robotic and traditional laparoscopic surgeries , 2014, Surgical Endoscopy.

[48]  A Prospective, Observational, Multicentre Study Concerning Nontechnical Skills in Robot-assisted Radical Cystectomy Versus Open Radical Cystectomy , 2020, European urology open science.

[49]  Gregory Wilding,et al.  Cognitive skills assessment during robot‐assisted surgery: separating the wheat from the chaff , 2015, BJU international.

[50]  David Azari,et al.  In Search of Characterizing Surgical Skill. , 2019, Journal of surgical education.

[51]  J L Ochsner,et al.  Minimally invasive surgical procedures. , 2000, The Ochsner journal.

[52]  R. Darin Ellis,et al.  Toward Personalized Training and Skill Assessment in Robotic Minimally Invasive Surgery , 2016, WCE 2016.

[53]  Teresa Wilcox,et al.  fNIRS in the developmental sciences. , 2015, Wiley interdisciplinary reviews. Cognitive science.

[54]  Lora Cavuoto,et al.  Anticipation, teamwork and cognitive load: chasing efficiency during robot-assisted surgery , 2017, BMJ Quality & Safety.

[55]  A. Darzi,et al.  Qualitative and quantitative analysis of the learning curve of a simulated surgical task on the da Vinci system , 2004, Surgical Endoscopy And Other Interventional Techniques.

[56]  Probabilistic Method to Improve the Accuracy of Computer-Integrated Surgical Systems , 2019, Acta Polytechnica Hungarica.

[57]  J. Pow-Sang,et al.  Urology residents experience comparable workload profiles when performing live porcine nephrectomies and robotic surgery virtual reality training modules , 2016, Journal of Robotic Surgery.

[58]  C. Thompson,et al.  Robot-assisted endoscopic submucosal dissection versus conventional ESD for colorectal lesions: outcomes of a randomized pilot study in endoscopists without prior ESD experience (with video). , 2019, Gastrointestinal endoscopy.

[59]  Esther Lau,et al.  Impact of robotic assistance on mental workload and cognitive performance of surgical trainees performing a complex minimally invasive suturing task , 2019, Surgical Endoscopy.

[60]  A. Hung,et al.  The Importance of Technical and Non-technical Skills in Robotic Surgery Training. , 2018, European urology focus.

[61]  Nick Sevdalis,et al.  The impact of intra-operative interruptions on surgeons’ perceived workload: an observational study in elective general and orthopedic surgery , 2014, Surgical Endoscopy.

[62]  Hee Chan Kim,et al.  A Novel Wearable EEG and ECG Recording System for Stress Assessment , 2019, Sensors.

[63]  Mark R. Wilson,et al.  Conscious motor processing and movement self-consciousness: two dimensions of personality that influence laparoscopic training. , 2014, Journal of surgical education.

[64]  Rajnikant V. Patel,et al.  Robotics-Assisted Surgical Skills Evaluation based on Electrocortical Activity , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[65]  A. Darzi,et al.  Robotic Surgery Improves Technical Performance and Enhances Prefrontal Activation During High Temporal Demand , 2018, Annals of Biomedical Engineering.

[66]  R. Nager,et al.  Skin temperature reveals the intensity of acute stress , 2015, Physiology & Behavior.

[67]  W. Kinlaw,et al.  Indirect measurement of isovolumetric contraction time and tension period in normal subjects. , 1962, The American journal of cardiology.

[68]  V. Naik,et al.  Do technical skills correlate with non-technical skills in crisis resource management: a simulation study , 2012, British journal of anaesthesia.

[69]  J. Anger,et al.  Safety, efficiency and learning curves in robotic surgery: a human factors analysis , 2016, Surgical Endoscopy.

[70]  Pawel Wisz,et al.  Training in robotic surgery, replicating the airline industry. How far have we come? , 2019, World Journal of Urology.

[71]  Steven Yule,et al.  Surgeons' non-technical skills. , 2012, The Surgical clinics of North America.

[72]  Sandra G. Hart,et al.  Nasa-Task Load Index (NASA-TLX); 20 Years Later , 2006 .

[73]  Chandru P Sundaram,et al.  Validation of a novel virtual reality robotic simulator. , 2009, Journal of endourology.

[74]  Gordon H Guyatt,et al.  GrADe : what is “ quality of evidence ” and why is it important to clinicians ? rATING quALITY of evIDeNCe AND STreNGTH of reCommeNDATIoNS , 2022 .

[75]  The Value of Open Conversion Simulations During Robot-Assisted Radical Prostatectomy: Implications for Robotic Training Curricula. , 2015, Journal of endourology.

[76]  Nick Sevdalis,et al.  Reliability of a revised NOTECHS scale for use in surgical teams. , 2008, American journal of surgery.

[77]  Irfan A. Essa,et al.  Automated surgical skill assessment in RMIS training , 2017, International Journal of Computer Assisted Radiology and Surgery.

[78]  Masakatsu G. Fujie,et al.  Pilot study on verification of effectiveness on operability of assistance system for robotic tele-surgery using simulation , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[79]  M. Stella Atkins,et al.  Eye gaze patterns differentiate novice and experts in a virtual laparoscopic surgery training environment , 2004, ETRA.

[80]  Algis Daktariunas,et al.  Functional near-infrared spectroscopy: a continuous wave type based system for human frontal lobe studies , 2015, EXCLI journal.

[81]  G. Youngson.  Nontechnical skills in pediatric surgery: Factors influencing operative performance. , 2016, Journal of pediatric surgery.

[82]  Sandra Marshall,et al.  Using objective robotic automated performance metrics and task-evoked pupillary response to distinguish surgeon expertise , 2019, World Journal of Urology.

[83]  M. Forsman,et al.  Intraoperative workload in robotic surgery assessed by wearable motion tracking sensors and questionnaires , 2017, Surgical Endoscopy.

[84]  Stefanie Speidel,et al.  Video-based surgical skill assessment using 3D convolutional neural networks , 2019, International Journal of Computer Assisted Radiology and Surgery.

[85]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[86]  Lora Cavuoto,et al.  The Loud Surgeon Behind the Console: Understanding Team Activities During Robot-Assisted Surgery. , 2016, Journal of surgical education.

[87]  Renaldo C. Blocker,et al.  NASA-Task Load Index Differentiates Surgical Approach: Opportunities for Improvement in Colon and Rectal Surgery. , 2020, Annals of surgery.

[88]  A. Darzi,et al.  Inattention blindness in surgery , 2015, Surgical Endoscopy.

[89]  Nick Sevdalis,et al.  Observational teamwork assessment for surgery: content validation and tool refinement. , 2011, Journal of the American College of Surgeons.

[90]  J. Korndorffer,et al.  Robotic assistance improves intracorporeal suturing performance and safety in the operating room while decreasing operator workload , 2010, Surgical Endoscopy.

[91]  W. B. Seales,et al.  Assessing Mental Workload During Laparoscopic Surgery , 2005, Surgical innovation.

[92]  Gyusung I. Lee,et al.  Can a virtual reality surgical simulation training provide a self-driven and mentor-free skills learning? Investigation of the practical influence of the performance metrics from the virtual reality robotic surgery simulator on the skill learning and associated cognitive workloads , 2017, Surgical Endoscopy.

[93]  Paolo Fiorini,et al.  Neurophysiological measures for users' training objective assessment during simulated robot-assisted laparoscopic surgery , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[94]  Guang-Zhong Yang,et al.  Gaze-Contingent Motor Channelling, haptic constraints and associated cognitive demand for robotic MIS , 2012, Medical Image Anal..

[95]  H. John,et al.  Da Vinci© Skills Simulator™: is an early selection of talented console surgeons possible? , 2016, Journal of Robotic Surgery.

[96]  M. Stella Atkins,et al.  Workload assessment of surgeons: correlation between NASA TLX and blinks , 2012, Surgical Endoscopy.

[97]  William S. Helton,et al.  Validation of a Short Stress State Questionnaire , 2004 .

[98]  J. Heemskerk,et al.  Relax, It's Just Laparoscopy! A Prospective Randomized Trial on Heart Rate Variability of the Surgeon in Robot-Assisted versus Conventional Laparoscopic Cholecystectomy , 2014, Digestive Surgery.

[99]  Ahmed M. Zihni,et al.  Operative performance outcomes of a simulator-based robotic surgical skills curriculum , 2019, Surgical Endoscopy.

[100]  David B. Boles,et al.  The Multiple Resources Questionnaire (MRQ) , 2001 .

[101]  Juan Pablo Wachs,et al.  Joint Surgeon Attributes Estimation in Robot-Assisted Surgery , 2018, HRI.

[102]  Tamás Haidegger,et al.  Handover Process of Autonomous Vehicles – Technology and Application Challenges , 2019, Acta Polytechnica Hungarica.

[103]  K. Catchpole,et al.  Intra-operative disruptions, surgeon’s mental workload, and technical performance in a full-scale simulated procedure , 2015, Surgical Endoscopy.

[104]  L. Panait,et al.  Do laparoscopic skills transfer to robotic surgery? , 2014, The Journal of surgical research.

[105]  Khurshid A Guru,et al.  Technical mentorship during robot‐assisted surgery: a cognitive analysis , 2016, BJU international.

[106]  T. Haidegger,et al.  Robot-Assisted Minimally Invasive Surgical Skill Assessment—Manual and Automated Platforms , 2019, Acta Polytechnica Hungarica.

[107]  P. McCulloch,et al.  The influence of non-technical performance on technical outcome in laparoscopic cholecystectomy , 2007, Surgical Endoscopy.

[108]  M. Riley,et al.  Performance, Stress, Workload, and Coping Profiles in 1st Year Medical Students' Interaction with the Endoscopic/Laparoscopic and Robot-Assisted Surgical Techniques , 2008 .

[109]  M. Hallbeck,et al.  Impact of single-incision laparoscopic cholecystectomy (SILC) versus conventional laparoscopic cholecystectomy (CLC) procedures on surgeon stress and workload: a randomized controlled trial , 2015, Surgical Endoscopy.

[110]  T. Haidegger,et al.  Surgery in space: the future of robotic telesurgery , 2011, Surgical Endoscopy.

[111]  Joel S. Warm,et al.  Perceived Mental Workload in an Endocopic Surgery Simulator , 2005 .

[112]  R. Aggarwal,et al.  Non-technical skills assessment in surgery. , 2011, Surgical oncology.

[113]  Warren D. Smith,et al.  An ergonomic comparison of robotic and laparoscopic technique: the influence of surgeon experience and task complexity. , 2003, The Journal of surgical research.

[114]  Nabeel A. Arain,et al.  Proficiency-based training for robotic surgery: construct validity, workload, and expert levels for nine inanimate exercises , 2012, Surgical Endoscopy.

[115]  F. Shaffer,et al.  An Overview of Heart Rate Variability Metrics and Norms , 2017, Front. Public Health.

[116]  Chris Melhuish,et al.  Estimation of Tool-Tissue Forces in Robot-Assisted Minimally Invasive Surgery Using Neural Networks , 2019, Front. Robot. AI.

[117]  Manuela Perez,et al.  Comparative analysis of the functionality of simulators of the da Vinci surgical robot , 2015, Surgical Endoscopy.

[118]  Tian Zhou,et al.  Eye-Tracking Metrics Predict Perceived Workload in Robotic Surgical Skills Training , 2019, Hum. Factors.

[119]  Omaira Rodríguez,et al.  Robotic surgery training: construct validity of Global Evaluative Assessment of Robotic Skills (GEARS) , 2016, Journal of Robotic Surgery.

[120]  Imre J. Rudas,et al.  Employing Process Models for Surgical Training , 2020, 2020 IEEE 18th World Symposium on Applied Machine Intelligence and Informatics (SAMI).

[121]  Khurshid A Guru,et al.  Dynamic changes of brain functional states during surgical skill acquisition , 2018, PloS one.