Virtual Reality Simulator Training for Laparoscopic Colectomy: What Metrics Have Construct Validity?

BACKGROUND: Virtual reality simulation for laparoscopic colectomy has been used for training of surgical residents and has been considered as a model for technical skills assessment of board-eligible colorectal surgeons. However, construct validity (the ability to distinguish between skill levels) must be confirmed before widespread implementation. OBJECTIVE: This study was designed to specifically determine which metrics for laparoscopic sigmoid colectomy have evidence of construct validity. DESIGN: General surgeons that had performed fewer than 30 laparoscopic colon resections and laparoscopic colorectal experts (>200 laparoscopic colon resections) performed laparoscopic sigmoid colectomy on the LAP Mentor model. All participants received a 15-minute instructional warm-up and had never used the simulator before the study. Performance was then compared between each group for 21 metrics (procedural, 14; intraoperative errors, 7) to determine specifically which measurements demonstrate construct validity. Performance was compared with the Mann-Whitney U-test (p < 0.05 was significant). RESULTS: Fifty-three surgeons; 29 general surgeons, and 24 colorectal surgeons enrolled in the study. The virtual reality simulators for laparoscopic sigmoid colectomy demonstrated construct validity for 8 of 14 procedural metrics by distinguishing levels of surgical experience (p < 0.05). The most discriminatory procedural metrics (p < 0.01) favoring experts were reduced instrument path length, accuracy of the peritoneal/medial mobilization, and dissection of the inferior mesenteric artery. Intraoperative errors were not discriminatory for most metrics and favored general surgeons for colonic wall injury (general surgeons, 0.7; colorectal surgeons, 3.5; p = 0.045). LIMITATIONS: Individual variability within the general surgeon and colorectal surgeon groups was not accounted for. CONCLUSIONS: The virtual reality simulators for laparoscopic sigmoid colectomy demonstrated construct validity for 8 procedure-specific metrics. However, using virtual reality simulator metrics to detect intraoperative errors did not discriminate between groups. If the virtual reality simulator continues to be used for the technical assessment of trainees and board-eligible surgeons, the evaluation of performance should be limited to procedural metrics.

[1]  Antony J. Hodgson,et al.  Intraoperative monitoring of laparoscopic skill development based on quantitative measures , 2009, Surgical Endoscopy.

[2]  J. Dankelman,et al.  Transfer validity of laparoscopic knot-tying training on a VR simulator to a realistic environment: A randomized controlled trial , 2008, Surgical Endoscopy.

[3]  T. Grantcharov,et al.  Randomized clinical trial of virtual reality simulation for laparoscopic skills training , 2004, The British journal of surgery.

[4]  C P Delaney,et al.  Assessment of Comparative Skills Between Hand-Assisted and Straight Laparoscopic Colorectal Training on an Augmented Reality Simulator , 2010, Diseases of the colon and rectum.

[5]  King-Jen Chang,et al.  Laparoscopic Pelvic Autonomic Nerve-Preserving Surgery for Sigmoid Colon Cancer , 2008, Annals of surgical oncology.

[6]  A. Mclarty,et al.  Simulated Laparoscopic Sigmoidectomy Training: Responsiveness of Surgery Residents , 2009, Diseases of the colon and rectum.

[7]  R. Reznick,et al.  A Novel Approach to Assessing Technical Competence of Colorectal Surgery Residents: The Development and Evaluation of the Colorectal Objective Structured Assessment of Technical Skill (COSATS) , 2013, Annals of surgery.

[8]  F. Leblanc,et al.  A comparison of human cadaver and augmented reality simulator models for straight laparoscopic colorectal skills acquisition training. , 2010, Journal of the American College of Surgeons.

[9]  Paul G Gauger,et al.  Laparoscopic Skills Are Improved With LapMentor™ Training: Results of a Randomized, Double-Blinded Study , 2006, Annals of surgery.

[10]  M P Schijven,et al.  Determination of face validity for the Simbionix LAP mentor virtual reality training module. , 2006, Studies in health technology and informatics.

[11]  Aimin Zhang,et al.  Construct validity testing of a laparoscopic surgery simulator (Lap Mentor®) , 2008, Surgical Endoscopy.

[12]  M. Schijven,et al.  Construct validity of the LapSim: Can the LapSim virtual reality simulator distinguish between novices and experts? , 2007, Surgical Endoscopy.

[13]  T. Sawada,et al.  Impact of Autonomic Nerve Preservation and Lateral Node Dissection on Male Urogenital Function after Total Mesorectal Excision for Lower Rectal Cancer , 2006, World Journal of Surgery.

[14]  Lucian Panait,et al.  Designing and validating a customized virtual reality-based laparoscopic skills curriculum. , 2008, Journal of surgical education.

[15]  Paul G Gauger,et al.  LapMentor Metrics Possess Limited Construct Validity , 2008, Simulation in healthcare : journal of the Society for Simulation in Healthcare.

[16]  Anthony G. Gallagher,et al.  Construct validation of a novel hybrid virtual-reality simulator for training and assessing laparoscopic colectomy; results from the first course for experienced senior laparoscopic surgeons , 2008, Surgical Endoscopy.

[17]  Ethan D Grober,et al.  Intraoperative assessment of technical skills on live patients using economy of hand motion: establishing learning curves of surgical competence. , 2010, American journal of surgery.

[18]  R. Clayman,et al.  Construct validity testing of a laparoscopic surgical simulator. , 2006, Journal of the American College of Surgeons.

[19]  Rajesh Aggarwal,et al.  Proving the Effectiveness of Virtual Reality Simulation for Training in Laparoscopic Surgery , 2007, Annals of surgery.

[20]  Technical skills assessment as part of the selection process for a fellowship in minimally invasive surgery , 2009, Surgical Endoscopy.

[21]  Makoto Hashizume,et al.  Construct validity for eye–hand coordination skill on a virtual reality laparoscopic surgical simulator , 2007, Surgical Endoscopy.