Assessing the Surgeon's Technical Skills: Analysis of the Available Tools

The concept of assessing competency in surgical practice is not new and has taken on an added urgency in view of the recent high-profile inquiries into “botched cases” involving surgeons of various levels in different parts of the world. Until very recently, surgeons in the United Kingdom and other parts of the world, although required to undergo formal and compulsory examinations to test their factual knowledge and decision making, were not required to demonstrate technical ability. Therefore, there existed (and still exist) no objective assessment criteria to test trainees' surgical skill, especially during the exit examination, which, if passed, provides unrestricted license to surgeons to practice their specialties. However, with the introduction of a new curriculum by various surgical societies and a demand from the lay community for better standards, new assessment tools are emerging that focus on technical competency and that could objectively and reliably measure surgical skills. Furthermore, training authorities and hospitals are keen to embrace these changes for satisfactory accreditation and reaccreditation processes and to assure the public of the safety of the public and private health care systems. In the United Kingdom, two new surgical tools (Surgical Direct Observation of Procedural Skill, and Procedure Based Assessments) have been simultaneously introduced to assess surgical trainees. The authors describe these two assessment methods, provide an overview of other assessment tools currently or previously used to assess surgical skills, critically analyze the two new assessment tools, and reflect on the merit of simultaneously introducing them.

[1]  Teodor P Grantcharov,et al.  Assessment of technical surgical skills. , 2002, The European journal of surgery = Acta chirurgica.

[2]  Burkhard Schafer,et al.  All changed, changed utterly? , 2011, Datenschutz und Datensicherheit - DuD.

[3]  R Reznick,et al.  Using operative outcome to assess technical skill. , 2000, American journal of surgery.

[4]  R. Rege,et al.  Measuring operative performance after laparoscopic skills training: edited videotape versus direct observation. , 2000, Journal of laparoendoscopic & advanced surgical techniques. Part A.

[5]  A Darzi,et al.  The challenge of objective assessment of surgical skill. , 2001, American journal of surgery.

[6]  R Reznick,et al.  Exporting a technical skills evaluation technology to other sites. , 2001, American journal of surgery.

[7]  W. Wade,et al.  Assessing the performance of doctors in training. , 2003, BMJ.

[8]  A. Darzi,et al.  The use of electromagnetic motion tracking analysis to objectively measure open surgical skill in the laboratory-based model. , 2001, Journal of the American College of Surgeons.

[9]  D. Rowley,et al.  Assessment of performance in orthopaedic training. , 2005, The Journal of bone and joint surgery. British volume.

[10]  R. Reznick,et al.  Testing technical skill via an innovative "bench station" examination. , 1997, American journal of surgery.

[11]  D. Bergqvist,et al.  The examination assessment of technical competence in vascular surgery. , 2007, The British journal of surgery.

[12]  R. Reznick,et al.  Validation of an objective structured assessment of technical skill for surgical residents , 1996, Academic medicine : journal of the Association of American Medical Colleges.

[13]  A. Darzi,et al.  Objective assessment of technical skills in surgery , 2003, BMJ : British Medical Journal.

[14]  C. Roberts,et al.  Practical experience of using directly observed procedures, mini clinical evaluation examinations, and peer observation in pre-registration house officer (FY1) trainees , 2006, Postgraduate Medical Journal.

[15]  Alija Kulenović,et al.  Standards for Educational and Psychological Testing , 1999 .

[16]  D. Newble,et al.  Assessing the technical skills of surgical trainees , 2005, The British journal of surgery.

[17]  R. Satava,et al.  Virtual reality as a metric for the assessment of laparoscopic psychomotor skills , 2002, Surgical Endoscopy And Other Interventional Techniques.

[18]  R. Reznick,et al.  Reliability and construct validity of a structured technical skills assessment form. , 1994, American journal of surgery.

[19]  L W T Schuwirth,et al.  The use of clinical simulations in assessment , 2003, Medical education.

[20]  R. Harden,et al.  Assessment of clinical competence using objective structured examination. , 1975, British medical journal.

[21]  J. Pepper,et al.  No time to train the surgeons , 2004, BMJ : British Medical Journal.

[22]  K. Calman,et al.  Hospital doctors: training for the future , 1995, British journal of obstetrics and gynaecology.

[23]  J. Norcini,et al.  The Mini-CEX: A Method for Assessing Clinical Skills , 2003, Annals of Internal Medicine.

[24]  A. Cuschieri,et al.  Errors enacted during endoscopic surgery--a human reliability analysis. , 1998, Applied ergonomics.

[25]  L P Mandel,et al.  TEACHING AND EVALUATING SURGICAL SKILLS , 2000, Obstetrics and gynecology.

[26]  Peter Williams,et al.  Quality and Standards in Higher Education. , 2010 .

[27]  Melina C Vassiliou,et al.  A global assessment tool for evaluation of intraoperative laparoscopic skills. , 2005, American journal of surgery.

[28]  R. Reznick,et al.  Assessing competency in surgery: where to begin? , 2004, Surgery.

[29]  G. Joughin,et al.  Oral assessment and postgraduate medical examinations: establishing conditions for validity, reliability and fairness , 2010, Advances in health sciences education : theory and practice.

[30]  Heather Fry,et al.  A handbook of teaching & learning in higher education , 1999 .

[31]  R. Reznick,et al.  Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE‐format examination , 1998, Academic medicine : journal of the Association of American Medical Colleges.

[32]  A. Cuschieri,et al.  Competence Assessment of Laparoscopic Operative and Cognitive Skills: Objective Structured Clinical Examination (OSCE) or Observational Clinical Human Reliability Assessment (OCHRA) , 2006, World Journal of Surgery.

[33]  Daniel B. Jones,et al.  Laparoscopic training on bench models: better and more cost effective than operating room experience? , 2000, Journal of the American College of Surgeons.

[34]  J. D. Greaves,et al.  Watching anaesthetists work: using the professional judgement of consultants to assess the developing clinical competence of trainees. , 2000, British journal of anaesthesia.

[35]  Heather Fry,et al.  A Handbook for Teaching and Learning in Higher Education : Enhancing Academic Practice , 1999 .

[36]  W. Cordell,et al.  An evaluation of emergency medicine resident interaction time with faculty in different teaching venues. , 2004, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[37]  W. Wallace,et al.  Influencing the national training agenda. The UK & Ireland orthopaedic eLogbook. , 2005, The Journal of bone and joint surgery. British volume.

[38]  Alfred Cuschieri,et al.  The performance of master surgeons on the Advanced Dundee Endoscopic Psychomotor Tester: contrast validity study. , 2002, Archives of surgery.

[39]  J. Norcini,et al.  The Mini-CEX (Clinical Evaluation Exercise): A Preliminary Investigation , 1995, Annals of Internal Medicine.

[40]  R. Reznick,et al.  Objective structured assessment of technical skill (OSATS) for surgical residents , 1997, The British journal of surgery.

[41]  G. Miller The assessment of clinical skills/competence/performance , 1990, Academic medicine : journal of the Association of American Medical Colleges.

[42]  R Smith,et al.  All changed, changed utterly , 1998, BMJ.

[43]  J. Beard,et al.  Assessment of operative competence during carotid endarterectomy , 2007, The British journal of surgery.

[44]  J. Kopta,et al.  An approach to the evaluation of operative skills. , 1971, Surgery.

[45]  Andrew A Gumbs,et al.  Evaluation of resident laparoscopic performance using global operative assessment of laparoscopic skills. , 2007, Journal of the American College of Surgeons.

[46]  K. Vosburgh,et al.  Assessing Technical Skill in Surgery and Endoscopy: A Set of Metrics and an Algorithm (C-PASS) to Assess Skills in Surgical and Endoscopic Procedures , 2007, Surgical innovation.

[47]  A. Darzi,et al.  Assessing operative skill , 1999, BMJ.

[48]  I L Naylor,et al.  The qualities and conduct of an English surgeon in 1446: as described in a manuscript attributed to Thomas Morstede. , 1997, Annals of the Royal College of Surgeons of England.

[49]  H. Vorum,et al.  Vascular Society of Great Britain and Ireland , 2008 .

[50]  C. W. Cummings,et al.  Competency in surgical residency training: defining and raising the bar. , 2007, Academic medicine : journal of the Association of American Medical Colleges.

[51]  R. Reznick,et al.  Teaching and testing technical skills. , 1993, American journal of surgery.