The assessment of medical competencies.

Abstract Objective To describe the most widely used tools in the assessment of medical competencies, analyze their prevalence of use, their advantages and disadvantages and propose an appropriate model for our context. Methods We conducted a narrative review of articles from MEDLINE, following the PRISM protocol, and analyzed a total of 62 articles. Results The assessment of competencies is heterogeneous, especially in the educational and professional settings. The specific and technical competencies acquired during university education are mainly assessed using the objective structured clinical assessment. In the professional setting, core competencies are assessed using the 360° technique. Conclusions We need a rigorous empiric comparison of the efficiency of the tools according to the type of competency. We propose a competency management model for the “undergraduate/graduate/active professional” continuum, whose goal is to improve training and professional practice and thereby increase the quality of patient care.

[1]  Laura J. Morrison,et al.  Teaching and assessing resident competence in practice-based learning and improvement , 2004, Journal of General Internal Medicine.

[2]  C. Violato,et al.  Assessing Psychologists in Practice: Lessons From the Health Professions Using Multisource Feedback , 2013 .

[3]  R. Sellei,et al.  Peer teaching: a randomised controlled trial using student‐teachers to teach musculoskeletal ultrasound , 2010, Medical education.

[4]  G. Bergus,et al.  The reliability of summative judgements based on objective structured clinical examination cases distributed across the clinical year , 2007, Medical education.

[5]  Z. Zaidi,et al.  Revising an objective structured clinical examination in a resource-limited Pakistani Medical School. , 2009, Education for health.

[6]  C. Rees,et al.  The acceptability of 360‐degree judgements as a method of assessing undergraduate medical students' personal and professional behaviours , 2005, Medical education.

[7]  Steven R Simon,et al.  The relationship between second-year medical students' OSCE scores and USMLE Step 2 scores. , 2007, Journal of evaluation in clinical practice.

[8]  T. Wilkinson,et al.  Patients' global ratings of student competence. Unreliable contamination or gold standard? , 2002, Medical Education.

[9]  Rose Hatala,et al.  Assessing cardiac physical examination skills using simulation technology and real patients: a comparison study , 2008, Medical education.

[10]  J. Norcini,et al.  Implementation of competency‐based medical education: are we addressing the concerns and challenges? , 2015, Medical education.

[11]  B. Hodges,et al.  Conceptual and practical challenges in the assessment of physician competencies , 2015, Medical teacher.

[12]  J. Archer,et al.  Factors that might undermine the validity of patient and multi‐source feedback , 2011, Medical education.

[13]  K. Mann,et al.  Exploring family physicians' reactions to multisource feedback: perceptions of credibility and usefulness , 2005, Medical education.

[14]  360 degree assessment (multisource feedback) of UK trainee doctors: Field testing of team assessment of behaviours (TAB) , 2007, Medical teacher.

[15]  J S Gonnella,et al.  Empathy in medical students as related to academic performance, clinical competence and gender , 2002, Medical education.

[16]  J. Archer,et al.  Impact of workplace based assessment on doctors’ education and performance: a systematic review , 2010, BMJ : British Medical Journal.

[17]  A. Reece,et al.  Competency domains in an undergraduate Objective Structured Clinical Examination: their impact on compensatory standard setting , 2008, Medical education.

[18]  Sara A. Pyle,et al.  The pediatric resident training on tobacco project: interim findings. , 2006, Journal of the National Medical Association.

[19]  E. Hundert,et al.  Defining and assessing professional competence. , 2002, JAMA.

[20]  C. Nikendei,et al.  Effects of basic clinical skills training on objective structured clinical examination performance , 2005, Medical education.

[21]  John G Frohna,et al.  A computer-based OSCE station to measure competence in evidence-based medicine skills in medical students. , 2002, Academic medicine : journal of the Association of American Medical Colleges.

[22]  A. Evans,et al.  Teaching cardiac examination skills , 2007, Journal of General Internal Medicine.

[23]  B. Brett,et al.  Multisite Geriatrics Clerkship for Fourth‐Year Medical Students: A Successful Model for Teaching the Association of American Medical Colleges' Core Competencies , 2009, Journal of the American Geriatrics Society.

[24]  B. Jolly,et al.  Predictive validity and estimated cut score of an objective structured clinical examination (OSCE) used as an assessment of clinical skills at the end of the first clinical year , 2002, Medical education.

[25]  Eiji Yano,et al.  Can nonverbal communication skills be taught? , 2010, Medical teacher.

[26]  G. Alexander,et al.  Assessing competence of residents to discuss end-of-life issues. , 2005, Journal of palliative medicine.

[27]  C. Violato,et al.  The Reliability, Validity, and Feasibility of Multisource Feedback Physician Assessment: A Systematic Review , 2014, Academic medicine : journal of the Association of American Medical Colleges.

[28]  Y T Wun,et al.  Communication skill of general practitioners: any room for improvement? how much can it be improved? , 2003, Medical education.

[29]  M. Mackay,et al.  Development, impact, and measurement of enhanced physical diagnosis skills , 2009, Advances in health sciences education : theory and practice.

[30]  Patricia O'Sullivan,et al.  Development and Implementation of an Objective Structured Clinical Examination to Provide Formative Feedback on Communication and Interpersonal Skills in Geriatric Training , 2008, Journal of the American Geriatrics Society.

[31]  R. Karani,et al.  Impact of a 360-degree Professionalism Assessment on Faculty Comfort and Skills in Feedback Delivery , 2008, Journal of General Internal Medicine.

[32]  Stephen Billett,et al.  Competency‐based medical education: origins, perspectives and potentialities , 2014, Medical education.

[33]  C. V. D. van der Vleuten,et al.  Challenges in multisource feedback: intended and unintended outcomes , 2007, Medical education.

[34]  P. Lelliott,et al.  Questionnaires for 360-degree assessment of consultant psychiatrists: Development and psychometric properties , 2008, British Journal of Psychiatry.

[35]  M. Dikici,et al.  Teaching medical students how to break bad news: A Turkish experience , 2009, Journal of Cancer Education.

[36]  C. Violato,et al.  Likelihood of Change: A Study Assessing Surgeon Use of Multisource Feedback Data , 2003, Teaching and learning in medicine.

[37]  J. Orlander,et al.  Comparing a Self-Administered Measure of Empathy with Observed Behavior Among Medical Students , 2010, Journal of general internal medicine.

[38]  D. Coady,et al.  Validation of MSAT: an instrument to measure medical students' self‐assessed confidence in musculoskeletal examination skills , 2007, Medical education.

[39]  M. Hojat,et al.  Students' personality and ratings of clinical competence in medical school clerkships: a longitudinal study , 2004 .

[40]  C. Violato,et al.  Self and Peer Assessment of Pediatricians, Psychiatrists and Medicine Specialists: Implications for Self-Directed Learning , 2006, Advances in health sciences education : theory and practice.

[41]  Peter J Hull,et al.  Optimal sequencing of bedside teaching and computer‐based learning: a randomised trial , 2009, Medical education.

[42]  C. Ananth,et al.  Evaluating Pelvic Examination Training: Does Faculty Involvement Make a Difference? A Randomized Controlled Trial , 2010, Teaching and learning in medicine.

[43]  R. Harden,et al.  Assessment of clinical competence using objective structured examination. , 1975, British medical journal.

[44]  J. Ioannidis,et al.  The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration , 2009, BMJ : British Medical Journal.

[45]  Wayne Woloschuk,et al.  Undesired variance due to examiner stringency/leniency effect in communication skill scores assessed in OSCEs , 2008, Advances in health sciences education : theory and practice.

[46]  L. Ungar,et al.  Ability of primary care physician's to break bad news: a performance based assessment of an educational intervention. , 2006, Patient education and counseling.

[47]  S. Thammasitboon,et al.  Exploring Individual Opinions of Potential Evaluators in a 360-Degree Assessment: Four Distinct Viewpoints of a Competent Resident , 2008, Teaching and learning in medicine.

[48]  A. Muijtjens,et al.  Unintentional failure to assess for experience in senior undergraduate OSCE scoring , 2008, Medical education.

[49]  P. Zuithoff,et al.  Predictive validity of a written knowledge test of skills for an OSCE in postgraduate training for general practice , 2002, Medical education.

[50]  Judy A. Shea,et al.  Resident Identification of Learning Objectives after Performing Self-assessment based upon the ACGME Core Competencies , 2008, Journal of General Internal Medicine.

[51]  Ana María Fernández Araque.  Competencias de las profesiones sanitarias , 2008 .

[52]  O. ten Cate,et al.  Senior medical students' appraisal of CanMEDS competencies , 2007, Medical education.

[53]  A. Kalet,et al.  Can interactive skills-based seminars with standardized patients enhance clinicians' prevention skills? Measuring the impact of a CME program. , 2010, Patient education and counseling.

[54]  M. Rieder,et al.  “But I See Old People Everywhere”: Dispelling the Myth That Eldercare Is Learned in Nongeriatric Clerkships , 2010, Academic medicine : journal of the Association of American Medical Colleges.

[55]  K. Hoffmann,et al.  Cross-year peer tutoring on internal medicine wards: Effects on self-assessed clinical competencies–A group control design study , 2009, Medical teacher.

[56]  Paul F. Wimmers,et al.  Influence of clerkship experiences on clinical competence , 2006, Medical education.

[57]  Johanna Shapiro,et al.  Point-of-view writing: A method for increasing medical students' empathy, identification and expression of emotion, and insight. , 2006, Education for health.

[58]  Dianne P. Wagner,et al.  The Patient Safety OSCE for PGY-1 Residents: A Centralized Response to the Challenge of Culture Change , 2009, Teaching and learning in medicine.

[59]  I. Koren,et al.  Objective structured clinical evaluation of clinical competence: an integrative review. , 2009, Journal of advanced nursing.

[60]  R. Grol,et al.  Doctors’ perceptions of why 360‐degree feedback does (not) work: a qualitative study , 2009, Medical education.

[61]  P. Bowie,et al.  Factors influencing the effectiveness of multisource feedback in improving the professional practice of medical doctors: a systematic review , 2014, BMC Medical Education.

[62]  B. Wilson.  Performance-based assessment of internal medicine interns: evaluation of baseline clinical and communication skills. , 2002, Academic medicine : journal of the Association of American Medical Colleges.

[63]  C. Violato,et al.  Assessment of Psychiatrists in Practice through Multisource Feedback , 2008, Canadian journal of psychiatry. Revue canadienne de psychiatrie.

[64]  J. Archer,et al.  Use of SPRAT for peer review of paediatricians in training , 2005, BMJ : British Medical Journal.

[65]  E. C. Corbett,et al.  Sharpening the Eye of the OSCE with Critical Action Analysis , 2008, Academic medicine : journal of the Association of American Medical Colleges.

[66]  C. V. D. van der Vleuten,et al.  Standardized or real patients to test clinical competence? The long case revisited , 2001, Medical education.

[67]  Lawrence G. Smith,et al.  The use of standardised patients to assess clinical competence: does practice make perfect? , 2006, Medical education.

[68]  A. Jefferies,et al.  Using an objective structured clinical examination (OSCE) to assess multiple physician competencies in postgraduate training , 2007, Medical teacher.

[69]  Matthew S. Prewett,et al.  A systematic review of the reliability of objective structured clinical examination scores , 2011, Medical education.

[70]  M. Short,et al.  Education Research: Evaluating acute altered mental status , 2008, Neurology.

[71]  J. Frank,et al.  The CanMEDS initiative: implementing an outcomes-based framework of physician competencies , 2007, Medical teacher.

[72]  C. Violato,et al.  The impact of a hybrid online and classroom-based course on palliative care competencies of family medicine residents , 2008, Palliative medicine.

[73]  Peter Henriksson,et al.  Active interprofessional education in a patient based setting increases perceived collaborative and professional competence , 2009, Medical teacher.

[74]  K. Rzechula,et al.  High fidelity simulation can discriminate between novice and experienced residents when assessing competency in patient care , 2007, Medical teacher.

[75]  Joel Tsevat,et al.  Modeling the effects of spirituality/religion on patients’ perceptions of living with HIV/AIDS , 2006, Journal of General Internal Medicine.

[76]  J. Spencer,et al.  Clarifying the concepts of confidence and competence to produce appropriate self‐evaluation measurement scales , 2000, Medical education.

[77]  Olle ten Cate,et al.  Entrustability of professional activities and competency‐based training , 2005 .

[78]  J. Norcini,et al.  Workplace-based assessment as an educational tool: AMEE Guide No. 31 , 2007, Medical teacher.

[79]  C. V. D. van der Vleuten,et al.  The assessment of professional competence: building blocks for theory development. , 2010, Best practice & research. Clinical obstetrics & gynaecology.

[80]  D. Kaufman,et al.  What makes a physician an exemplary communicator with patients? , 2007, Patient education and counseling.

[81]  Ronald M Epstein,et al.  Peer assessment of professional competence , 2005, Medical education.

[82]  Joseph Jaeger,et al.  Assessment of a 360-Degree Instrument to Evaluate Residents’ Competency in Interpersonal and Communication Skills , 2004, Academic medicine : journal of the Association of American Medical Colleges.

[83]  J. Perera,et al.  The use of objective structured self-assessment and peer-feedback (OSSP) for learning communication skills: evaluation using a controlled trial , 2010, Advances in health sciences education : theory and practice.