Objective structured assessment of technical skill in upper extremity surgery.

PURPOSE Objective assessment of technical skills in hand surgery has been lacking. This article reports on an Objective Structured Assessment of Technical Skills format of a multiple bench-station evaluation of orthopedic surgery residents' technical skills for 3 common upper extremity surgeries. METHODS Twenty-seven residents (6 postgraduate year [PGY] 2, 8 PGY 3, 8 PGY 4, and 5 PGY 5) participated in the examination. Each resident performed surgery on a cadaveric specimen at 3 stations, trigger finger release (TFR), open carpal tunnel release, and distal radius fracture fixation. A board-certified hand surgeon evaluated trainee performance at each station, using a procedure-specific detailed checklist, a validated global rating scale, and pass/fail assessment. A resident post-testing evaluation was collected. RESULTS Construct validity with correlation between year in training and detailed checklist scores was demonstrated for TFR and carpal tunnel release; between year in training and global rating scores for TFR and distal radius fracture fixation; and between year in training and pass/fail assessment for TFR. Criterion validity was demonstrated by the correlation between global rating scale scores, detailed checklist scores, and pass/fail assessment for TFR, carpal tunnel release, and distal radius fracture fixation. Time to complete the surgery was not correlated with surgical performance. Residents rated the multiple-station Objective Structured Assessment of Technical Skills format as highly educational. CONCLUSIONS This study reports that a surgeon's ability to release a trigger finger does not correlate specifically to his or her ability to perform a carpal tunnel release or to perform plate fixation of a radius fracture. The results of this study would indicate that, for 3 different surgical simulations representing procedures of varying complexity, assessments by a single assessment tool is not adequate. To completely understand a resident's abilities, assessment by checklist (understanding the steps of the surgery), global rating scales (assessment of basic surgical skills in light of lesser or greater complexity surgeries), and pass/fail assessment (examination of adverse events) are all necessary components. TYPE OF STUDY/LEVEL OF EVIDENCE Therapeutic II.

[1]  R. Reznick,et al.  Teaching surgical skills--changes in the wind. , 2006, The New England journal of medicine.

[2]  R. Reznick,et al.  Testing technical skill via an innovative "bench station" examination. , 1997, American journal of surgery.

[3]  Ara Darzi,et al.  Comparison of bench test evaluations of surgical skill with live operating performance assessments. , 2004, Journal of the American College of Surgeons.

[4]  K. A. Ericsson,et al.  Expert and exceptional performance: evidence of maximal adaptation to task constraints. , 1996, Annual review of psychology.

[5]  A. Dubrowski,et al.  The effect of computer navigation on trainee learning of surgical skills. , 2007, The Journal of bone and joint surgery. American volume.

[6]  R. Reznick,et al.  A comprehensive examination for senior surgical residents. , 2000, American journal of surgery.

[7]  L. Cronbach.  Coefficient alpha and the internal structure of tests , 1951 .

[8]  Richard Reznick,et al.  The effectiveness of video feedback in the acquisition of orthopedic technical skills. , 2004, American journal of surgery.

[9]  J. Kopta,et al.  An approach to the evaluation of operative skills. , 1971, Surgery.

[10]  M. Bridges,et al.  The financial impact of teaching surgical residents in the operating room. , 1999, American journal of surgery.

[11]  Adam Dubrowski,et al.  The contributions of kinesiology to surgical education. , 2004, The Journal of bone and joint surgery. American volume.

[12]  R. Reznick,et al.  Objective structured assessment of technical skill (OSATS) for surgical residents , 1997, The British journal of surgery.

[13]  B. Goff,et al.  A six-year study of surgical teaching and skills evaluation for obstetric/gynecologic residents in porcine and inanimate surgical models. , 2005, American journal of obstetrics and gynecology.

[14]  I A Anderson,et al.  A simulation-based training system for hip fracture fixation for use within the hospital environment. , 2007, Injury.

[15]  Tien-Tsin Wong,et al.  Virtual reality techniques. Application to anatomic visualization and orthopaedics training. , 2006, Clinical orthopaedics and related research.

[16]  Diane P. Martin,et al.  Musculoskeletal function assessment instrument: Criterion and construct validity , 1996, Journal of orthopaedic research : official publication of the Orthopaedic Research Society.

[17]  R. Reznick,et al.  Teaching and testing technical skills. , 1993, American journal of surgery.

[18]  J D Mabrey,et al.  A comparison of operative times in arthroscopic ACL reconstruction between orthopaedic faculty and residents: the financial impact of orthopaedic surgical training in the operating room. , 2001, The Iowa orthopaedic journal.

[19]  William E. GarrettJr..  American Board of Orthopaedic Surgery Practice of the Orthopaedic Surgeon: Part-II, Certification Examination Case Mix , 2006 .

[20]  M. Swiontkowski,et al.  American Board of Orthopaedic Surgery Practice of the Orthopaedic Surgeon: Part-II, certification examination case mix. , 2006, The Journal of bone and joint surgery. American volume.