ANALYSES/LITERATURE REVIEWS: Evaluating Competence Using a Portfolio: A Literature Review and Web-Based Application to the ACGME Competencies

Background: The Accreditation Council for Graduate Medical Education is shifting its focus from a structure and process system of graduate medical education to one that is outcomes based, requiring all residents in training to achieve competence in six broad domains and presenting educators with the challenge of evaluation. Summary: We searched the literature and reviewed all articles relating to portfolio assessment in medicine. In addition, we reviewed articles that addressed standard tools currently in use for resident performance assessment. The use of a portfolio allows one to incorporate a variety of assessment tools needed to evaluate the diverse domains of competence and also fosters reflective learning, which is key to professional development. From lessons learned through this review, we developed a Web-based evaluation portfolio for residency training that we describe here in the hope of facilitating this process for others. Conclusions: Web-based portfolio assessment provides an ideal venue for the evaluation of competence and has the ability to provide educators with a research infrastructure to practice evidence-based education.

[1]  M. Friedman Ben David,et al.  AMEE Medical Education Guide No. 24: Portfolios as a method of student assessment , 2001, Medical teacher.

[2]  E. Murray.  Challenges in educational research , 2002, Medical education.

[3]  T Dornan,et al.  Skills base: a Web-based electronic learning portfolio for clinical skills. , 2001, Academic medicine : journal of the Association of American Medical Colleges.

[4]  C. Carraccio,et al.  The objective structured clinical examination: a step in the direction of competency-based evaluation. , 2000, Archives of pediatrics & adolescent medicine.

[5]  N. Mathers,et al.  Portfolio‐based learning: continuing medical education for general practitioners — a mid‐point evaluation , 1997, Medical education.

[6]  David Snadden Mary Thomas,et al.  The use of portfolio learning in medical education , 1998 .

[7]  Enhancing reliability in portfolio assessment: 'shaping' the portfolio , 2001, Medical teacher.

[8]  C. Carraccio,et al.  Shifting Paradigms: From Flexner to Competencies , 2002, Academic medicine : journal of the Association of American Medical Colleges.

[9]  D. Schoen.  Educating the reflective practitioner , 1987 .

[10]  M. Fung,et al.  An Internet‐based learning portfolio in resident education: the KOALA™ multicentre programme , 2000, Medical education.

[11]  J. Spencer,et al.  Clarifying the concepts of confidence and competence to produce appropriate self‐evaluation measurement scales , 2000, Medical education.

[12]  Gail M. Jensen,et al.  Portfolios and Professional Development in the Health Professions , 1994 .

[13]  K. Lonka,et al.  Portfolios as a learning tool in obstetrics and gynaecology undergraduate training , 2001, Medical education.

[14]  E. Holmboe,et al.  Methods for Evaluating the Clinical Competence of Residents in Internal Medicine: A Review , 1998, Annals of Internal Medicine.

[15]  John Parboosingh,et al.  Learning portfolios: Potential to assist health professionals with self-directed learning , 1996 .

[16]  M.H. Davis,et al.  Portfolio assessment in medical students' final examinations , 2001, Medical teacher.

[17]  M. Dimatteo,et al.  Use of nonphysician staff to evaluate humanistic behavior of internal medicine residents and faculty members. , 1986, Journal of medical education.

[18]  E. Mazzaferri,et al.  Nurses as evaluators of the humanistic behavior of internal medicine residents. , 1987, Journal of medical education.

[19]  P. Jennett,et al.  Comparing peer and faculty evaluations in an internal medicine residency , 1994, Academic medicine : journal of the Association of American Medical Colleges.

[20]  Glenn Regehr,et al.  Measuring Self-assessment: Current State of the Art , 2002, Advances in health sciences education : theory and practice.

[21]  E. Altmaier,et al.  Defining successful performance among pediatric residents. , 1990, Pediatrics.

[22]  I G Finlay,et al.  A randomized controlled study of portfolio learning in undergraduate cancer education , 1998, Medical education.

[23]  N. Mathers,et al.  Portfolios in continuing medical education – effective and efficient? , 1999, Medical education.

[24]  R. M. Smith,et al.  The triple‐jump examination as an assessment tool in the problem‐based medical curriculum at the University of Hawaii , 1993, Academic medicine : journal of the Association of American Medical Colleges.

[25]  M. J. Gordon.  A review of the validity and accuracy of self‐assessments in health professions training , 1991, Academic medicine : journal of the Association of American Medical Colleges.

[26]  John Pitts,et al.  Enhancing reliability in portfolio assessment: discussions between assessors , 2002, Medical teacher.

[27]  D Snadden.  Portfolios – attempting to measure the unmeasurable? , 1999, Medical education.

[28]  J. Woolliscroft,et al.  Resident‐patient interactions: the humanistic qualities of internal medicine residents assessed by patients, attending physicians, program supervisors, and nurses , 1994, Academic medicine : journal of the Association of American Medical Colleges.

[29]  J. Pitts,et al.  Educational portfolios in the assessment of general practice trainers: reliability of assessors , 1999, Medical education.

[30]  C M Campbell,et al.  Study of physicians' use of a software program to create a portfolio of their self‐directed learning , 1996, Academic medicine : journal of the Association of American Medical Colleges.

[31]  David Newble,et al.  The use of portfolios for assessment of the competence and performance of doctors in practice , 2002, Medical education.

[32]  J. Norcini,et al.  The Mini-CEX (Clinical Evaluation Exercise): A Preliminary Investigation , 1995, Annals of Internal Medicine.