Assessing visual control during simulated and live operations: gathering evidence for the content validity of simulation using eye movement metrics

BackgroundAlthough virtual reality (VR) simulators serve an important role in the training and assessment of surgeons, they need to be evaluated for evidence of validity. Eye-tracking technology and measures of visual control have been used as an adjunct to the performance parameters produced by VR simulators to help in objectively establishing the construct validity (experts vs. novices) of VR simulators. However, determining the extent to which VR simulators represent the real procedure and environment (content validity) has largely been a subjective process undertaken by experienced surgeons. This study aimed to examine the content validity of a VR transurethral resection of the prostate (TURP) simulator by comparing visual control metrics taken during simulated and real TURP procedures.MethodsEye-tracking data were collected from seven surgeons performing 14 simulated TURP operations and three surgeons performing 15 real TURP operations on live patients. The data were analyzed offline, and visual control metrics (number and duration of fixations, percentage of time the surgeons fixated on the screen) were calculated.ResultsThe surgeons displayed more fixations of a shorter duration and spent less time fixating on the video monitor during the real TURP than during the simulated TURP. This could have been due to (1) the increased complexity of the operating room (OR) environment (2) the decreased quality of the image of the urethra and associated anatomy (compared with the VR simulator), or (3) the impairment of visual attentional control due to the increased levels of stress likely experienced in the OR.ConclusionsThe findings suggest that the complexity of the environment surrounding VR simulators needs to be considered in the design of effective simulated training curricula. The study also provides support for the use of eye-tracking technology to assess the content validity of simulation and to examine psychomotor processes during live operations.

[1]  Necip Berme,et al.  Eye motion parameters correlate with level of experience in video-assisted surgery: objective testing of three tasks. , 2005, Journal of laparoendoscopic & advanced surgical techniques. Part A.

[2]  Mark R. Wilson,et al.  Cheating experience: Guiding novices to adopt the gaze strategies of experts expedites the learning of technical laparoscopic skills. , 2012, Surgery.

[3]  A G Gallagher,et al.  Surgical simulation—a ‘good idea whose time has come’ , 2003, The British journal of surgery.

[4]  M. S. Atkins,et al.  Analysis of eye gaze: Do novice surgeons look at the same location as expert surgeons during a laparoscopic operation? , 2012, Surgical Endoscopy.

[5]  C. Lallas,et al.  Face, content, and construct validation of the da Vinci Skills Simulator. , 2012, Urology.

[6]  Mark R. Wilson,et al.  Psychomotor control in a virtual laparoscopic surgery training environment: gaze control parameters differentiate novices from experts , 2010, Surgical Endoscopy.

[7]  Guang-Zhong Yang,et al.  Collaborative Gaze Channelling for Improved Cooperation During Robotic Assisted Surgery , 2012, Annals of Biomedical Engineering.

[8]  P. Dasgupta,et al.  Current status of validation for robotic surgery simulators – a systematic review , 2013, BJU international.

[9]  A. Scherpbier,et al.  Validation and implementation of surgical simulators: a critical review of present, past, and future , 2009, Surgical Endoscopy.

[10]  Guang-Zhong Yang,et al.  Collaborative eye tracking: a potential training tool in laparoscopic surgery , 2012, Surgical Endoscopy.

[11]  Mark R. Wilson,et al.  Gaze training improves the retention and transfer of laparoscopic technical skills in novices , 2013, Surgical Endoscopy.

[12]  Mark R. Wilson,et al.  The influence of anxiety on visual attentional control in basketball free throw shooting. , 2009, Journal of sport & exercise psychology.

[13]  R. Säljö,et al.  Expertise Differences in the Comprehension of Visualizations: a Meta-Analysis of Eye-Tracking Research in Professional Domains , 2011 .

[14]  Mark R. Wilson,et al.  Face validity, construct validity and training benefits of a virtual reality TURP simulator. , 2012, International journal of surgery.

[15]  A. Gallagher,et al.  Augmenting, not cheating! , 2013, Surgery.

[16]  J. J. Jakimowicz,et al.  Consensus guidelines for validation of virtual reality surgical simulators , 2005, Surgical Endoscopy And Other Interventional Techniques.

[17]  Colin Swindells,et al.  Surgeon's vigilance in the operating room. , 2011, American journal of surgery.

[18]  Mark R. Wilson,et al.  You can't beat experience, but you can cheat it. , 2013, Surgery.

[19]  Michael J Schwartz,et al.  Eye Metrics as an Objective Assessment of Surgical Skill , 2010, Annals of surgery.

[20]  Mark R. Wilson,et al.  Perceptual impairment and psychomotor control in virtual laparoscopic surgery , 2011, Surgical Endoscopy.

[21]  M. Stella Atkins,et al.  Eye gaze patterns differentiate novice and experts in a virtual laparoscopic surgery training environment , 2004, ETRA.

[22]  A. Park,et al.  Gaze disruptions experienced by the laparoscopic operating surgeon , 2010, Surgical Endoscopy.

[23]  A. Gallagher,et al.  Virtual Reality Training in Laparoscopic Surgery: A Preliminary Assessment of Minimally Invasive Surgical Trainer Virtual Reality (MIST VR) , 1999, Endoscopy.