Validation of a global assessment of arthroscopic skills in a cadaveric knee model.

PURPOSE The purpose of this study was to determine whether a global assessment of arthroscopic skills was valid for blinded assessment of cadaveric diagnostic knee arthroscopy. METHODS A global skills assessment for arthroscopy was created using a published theory of the development of expertise. Faculty surgeons, fellows, and residents were consented and enrolled in this institutional review board-approved validation study. All participants were oriented to the equipment and procedures for diagnostic arthroscopy of the knee. After reviewing the anatomic structures to be visualized, participants were allowed 10 minutes to complete a diagnostic arthroscopy of the knee. The hands and arthroscopic view were recorded during this attempt. Resident participants completed a second filmed diagnostic arthroscopy 1 week after the initial attempt. Five blinded reviewers watched the synchronized videos and assessed arthroscopic skills with a procedure-specific checklist and the newly developed global skills assessment. The agreement between reviewers was determined by intraclass correlation coefficient. Internal consistency was determined with Cronbach's α. Test-retest reliability was measured by correlating repeated arthroscopies by residents. The ability of the global assessment to discriminate skill levels was determined with between-group Mann-Whitney U tests. RESULTS The agreement between global assessment scores was strong (I.C.C. = 0.80, 95% C.I. 0.68-0.92). The internal consistency of evaluations was excellent (Cronbach's α = 0.97), and the test-retest reliability was strong (r = 0.52). The global assessment score was shown to be able to discriminate between skill levels by an analysis of variance indicating the difference in means among the various levels of training (P < .0001). CONCLUSIONS The Objective Assessment of Arthroscopic Skills is a useful adjunct to arthroscopic educators and learners and could be used for in-training evaluations. CLINICAL RELEVANCE The Objective Assessment of Arthroscopic Skills is an instrument that can be employed to measure the impact of skills curricula, including but not limited to simulation.

[1]  R Poss,et al.  Development of a virtual reality arthroscopic knee simulator. , 2000, Studies in health technology and informatics.

[2]  Ara Darzi,et al.  Skills acquisition and assessment after a microsurgical skills course for ophthalmology residents. , 2009, Ophthalmology.

[3]  Sakti Srivastava,et al.  Initial evaluation of a shoulder arthroscopy simulator: establishing construct validity. , 2004, Journal of shoulder and elbow surgery.

[4]  L P Mandel,et al.  TEACHING AND EVALUATING SURGICAL SKILLS , 2000, Obstetrics and gynecology.

[5]  R B Loftin,et al.  Assessing a virtual reality surgical skills simulator. , 1996, Studies in health technology and informatics.

[6]  Jennifer M Wu,et al.  Objective structured assessment of technical skills for repair of fourth-degree perineal lacerations. , 2008, American journal of obstetrics and gynecology.

[7]  R. Reznick,et al.  Objective structured assessment of technical skill (OSATS) for surgical residents , 1997, The British journal of surgery.

[8]  R. Reznick,et al.  Validation of an objective structured assessment of technical skill for surgical residents , 1996, Academic medicine : journal of the Association of American Medical Colleges.

[9]  R. Reznick,et al.  Reliability and construct validity of a structured technical skills assessment form. , 1994, American journal of surgery.

[10]  S. Dreyfus,et al.  Peripheral Vision , 2005 .

[11]  R. Reznick,et al.  Teaching and testing technical skills. , 1993, American journal of surgery.

[12]  M. Ishii,et al.  Development and Pilot-Testing of a Feasible, Reliable, and Valid Operative Competency Assessment Tool for Endoscopic Sinus Surgery , 2009, American journal of rhinology & allergy.

[13]  Melina C Vassiliou,et al.  A global assessment tool for evaluation of intraoperative laparoscopic skills. , 2005, American journal of surgery.

[14]  Robert A Pedowitz,et al.  Evaluation of a virtual reality simulator for arthroscopy skills development. , 2002, Arthroscopy : the journal of arthroscopic & related surgery : official publication of the Arthroscopy Association of North America and the International Arthroscopy Association.

[15]  J L Rees,et al.  Retention of arthroscopic shoulder skills learned with use of a simulator. Demonstration of a learning curve and loss of performance level after a time delay. , 2009, The Journal of bone and joint surgery. American volume.

[16]  D. M. Binney,et al.  Value and face validity of objective structured assessment of technical skills (OSATS) for work based assessment of surgical skills in obstetrics and gynaecology , 2008, Medical teacher.

[17]  R. Arciero,et al.  The development of an objective model to assess arthroscopic performance. , 2009, The Journal of bone and joint surgery. American volume.

[18]  R. Reznick,et al.  Toward reliable operative assessment: the reliability and feasibility of videotaped assessment of laparoscopic technical skills , 2004, Surgical Endoscopy.

[19]  J. Norcini,et al.  The Mini-CEX: A Method for Assessing Clinical Skills , 2003, Annals of Internal Medicine.

[20]  D. Pratt,et al.  Orthopaedic surgeons as educators. Applying the principles of adult education to teaching orthopaedic residents. , 2007, The Journal of bone and joint surgery. American volume.

[21]  A. Darzi,et al.  Relationship between skill and outcome in the laboratory-based model. , 2002, Surgery.