Curriculum development for the workplace using Entrustable Professional Activities (EPAs): AMEE Guide No. 99

Abstract This Guide was written to support educators interested in building a competency-based workplace curriculum. It aims to provide an up-to-date overview of the literature on Entrustable Professional Activities (EPAs), supplemented with suggestions for practical application to curriculum construction, assessment and educational technology. The Guide first introduces concepts and definitions related to EPAs and then guidance for their identification, elaboration and validation, while clarifying common misunderstandings about EPAs. A matrix-mapping approach of combining EPAs with competencies is discussed, and related to existing concepts such as competency milestones. A specific section is devoted to entrustment decision-making as an inextricable part of working with EPAs. In using EPAs, assessment in the workplace is translated to entrustment decision-making for designated levels of permitted autonomy, ranging from acting under full supervision to providing supervision to a junior learner. A final section is devoted to the use of technology, including mobile devices and electronic portfolios to support feedback to trainees about their progress and to support entrustment decision-making by programme directors or clinical teams.

[1]  W. McGaghie,et al.  A critical review of simulation‐based medical education research: 2003–2009 , 2010, Medical education.

[2]  C. V. D. van der Vleuten,et al.  Programmatic assessment: From assessment of learning to assessment for learning , 2011, Medical teacher.

[3]  L. O. Dahle,et al.  Pros and cons of vertical integration between clinical medicine and basic science within a problem-based undergraduate medical curriculum: examples and experiences from Linköping, Sweden , 2002, Medical teacher.

[4]  M. J. O'neil,et al.  Nominal Group Technique: A process for initiating curriculum development in higher education , 1983 .

[5]  Christine Spratt,et al.  Using entrustable professional activities to guide curriculum development in psychiatry training , 2011, BMC medical education.

[6]  Thomas G. Cooney,et al.  The Development of Entrustable Professional Activities for Internal Medicine Residency Training: A Report From the Education Redesign Committee of the Alliance for Academic Internal Medicine , 2015, Academic medicine : journal of the Association of American Medical Colleges.

[7]  Marieke van der Schaaf,et al.  Vertical integration in medical school: effect on the transition to postgraduate training , 2010, Medical education.

[8]  E. Custers,et al.  Very long‐term retention of basic science knowledge in doctors after graduation , 2011, Medical education.

[9]  Olle Ten Cate,et al.  Building a competency-based workplace curriculum around entrustable professional activities: The case of physician assistant training , 2010, Medical teacher.

[10]  William F Iobst,et al.  Reconceptualizing variable rater assessments as both an educational and clinical care problem. , 2014, Academic medicine : journal of the Association of American Medical Colleges.

[11]  Janet Mee,et al.  What New Residents Do During Their Initial Months of Training , 2011, Academic medicine : journal of the Association of American Medical Colleges.

[12]  Brian C. George,et al.  Reliability, validity, and feasibility of the Zwisch scale for the assessment of intraoperative performance. , 2014, Journal of surgical education.

[13]  Joseph B Zwischenberger,et al.  A theory-based model for teaching and assessing residents in the operating room. , 2013, Journal of surgical education.

[14]  P. Remmelts,et al.  What are we preparing them for? Development of an inventory of tasks for medical, surgical and supportive specialties , 2013, Medical teacher.

[15]  Allan S. Detsky,et al.  Graded autonomy in medical education--managing things that go bump in the night. , 2014, The New England journal of medicine.

[16]  Geoffrey C. Williams,et al.  How self-determination theory can assist our understanding of the teaching and learning processes in medical education. AMEE Guide No. 59 , 2011, Medical teacher.

[17]  G. Regehr,et al.  Progressive Independence in Clinical Training: A Tradition Worth Defending? , 2005, Academic medicine : journal of the Association of American Medical Colleges.

[18]  Harold G. J. Bok.  Competency-based veterinary education: an integrative approach to learning and assessment in the clinical workplace , 2015, Perspectives on medical education.

[19]  D. Kolb.  Experiential Learning: Experience as the Source of Learning and Development , 1983 .

[20]  M A Albanese.  Challenges in using rater judgements in medical education. , 2000, Journal of evaluation in clinical practice.

[21]  Daniel C. West,et al.  The Promise and Challenge of Entrustable Professional Activities , 2014, Pediatrics.

[22]  Miriam Lacasse,et al.  The Application of Entrustable Professional Activities to Inform Competency Decisions in a Family Medicine Residency Program , 2015, Academic medicine : journal of the Association of American Medical Colleges.

[23]  S. B. Bierer,et al.  Evidence within a portfolio-based assessment program: What do medical students select to document their performance? , 2012, Medical teacher.

[24]  David C Aron,et al.  Measuring progressive independence with the resident supervision index: empirical approach. , 2010, Journal of Graduate Medical Education.

[25]  Trudie Roberts,et al.  Evaluating the utility of workplace-based assessment tools for speciality training. , 2010, Best practice & research. Clinical obstetrics & gynaecology.

[26]  Paul G Gauger,et al.  Entrustment of general surgery residents in the operating room: factors contributing to provision of resident autonomy. , 2014, Journal of the American College of Surgeons.

[27]  Henry E Fessler,et al.  Entrustable Professional Activities and Curricular Milestones for Fellowship Training in Pulmonary and Critical Care Medicine: Executive Summary from the Multi-Society Working Group , 2014, Critical care medicine.

[28]  Etienne Wenger,et al.  Situated Learning: Legitimate Peripheral Participation , 1991 .

[29]  Stephen Billett,et al.  Competency‐based medical education: origins, perspectives and potentialities , 2014, Medical education.

[30]  Karen Mann,et al.  Orienting Teaching Toward the Learning Process , 2004, Academic medicine : journal of the Association of American Medical Colleges.

[31]  Deborah Simpson,et al.  Toward a glossary of competency-based medical education terms. , 2014, Journal of graduate medical education.

[32]  Eric S Holmboe,et al.  Time to trust: longitudinal integrated clerkships and entrustable professional activities. , 2014, Academic medicine : journal of the Association of American Medical Colleges.

[33]  Stephen Richard Billett,et al.  Mimetic Learning in and for Work , 2014 .

[34]  Karen Malone,et al.  A critical time for medical education: the perils of competence-based reform of the curriculum , 2012, Advances in health sciences education : theory and practice.

[35]  Lorelei Lingard,et al.  Point-of-Care Assessment of Medical Trainee Competence for Independent Clinical Work , 2008, Academic medicine : journal of the Association of American Medical Colleges.

[36]  E. Hundert,et al.  Defining and assessing professional competence. , 2002, JAMA.

[37]  O. ten Cate,et al.  AM Last Page: What Entrustable Professional Activities Add to a Competency-Based Curriculum , 2014, Academic medicine : journal of the Association of American Medical Colleges.

[38]  M. Guha.  The Wiley-Blackwell Encyclopedia of Health, Illness, Behavior and Society , 2014 .

[39]  Hsm Helianthe Kort,et al.  Toepassing van het concept EPA als verbinding tussen professionele activiteiten en CanMEDS competentie-gebieden: pilotstudie Physician Assistant Urologie , 2008 .

[40]  Laura Mosqueda,et al.  What Is a Geriatrician? American Geriatrics Society and Association of Directors of Geriatric Academic Programs End‐of‐Training Entrustable Professional Activities for Geriatric Medicine , 2014, Journal of the American Geriatrics Society.

[41]  Julie G. Nyquist,et al.  Educating Physicians: A Call for Reform of Medical School and Residency , 2011 .

[42]  Vanessa C. Burch,et al.  Workplace-based assessment as an educational tool , 2007 .

[43]  Fedde Scheele,et al.  Assessment of competence and progressive independence in postgraduate clinical training , 2009, Medical education.

[44]  David C Aron,et al.  Measuring progressive independence with the resident supervision index: theoretical approach. , 2010, Journal of graduate medical education.

[45]  S. Lurie,et al.  Commentary: pitfalls in assessment of competency-based educational objectives. , 2011, Academic medicine : journal of the Association of American Medical Colleges.

[46]  D. Dunn,et al.  Experiential Learning , 2019, High Impact Teaching for Sport and Exercise Psychology Educators.

[47]  E. Custers,et al.  Analysing the concept of context in medical education , 2005, Medical education.

[48]  J. Glass,et al.  Competency based training is a framework for incompetence , 2014, BMJ : British Medical Journal.

[49]  C. Carraccio,et al.  Shifting Paradigms: From Flexner to Competencies , 2002, Academic medicine : journal of the Association of American Medical Colleges.

[50]  M. Talbot.  Monkey see, monkey do: a critique of the competency model in graduate medical education , 2004, Medical education.

[51]  Anna T Cianciolo,et al.  Behavioral specification of the entrustment process. , 2013, Journal of graduate medical education.

[52]  Jonathan Tolentino,et al.  Entrustment and Mapping of Observable Practice Activities for Resident Assessment , 2014, Journal of General Internal Medicine.

[53]  Carol Carraccio,et al.  Educational milestone development in the first 7 specialties to enter the next accreditation system. , 2013, Journal of graduate medical education.

[54]  O. ten Cate,et al.  Nuts and bolts of entrustable professional activities. , 2013, Journal of graduate medical education.

[55]  Janet Grant,et al.  The Incapacitating Effects of Competence: A Critique , 1999, Advances in health sciences education : theory and practice.

[56]  Elke Gruber,et al.  Learning at the workplace , 2009 .

[57]  O. ten Cate,et al.  Entrustability of professional activities and competency‐based training , 2005, Medical education.

[58]  John G. Hedberg,et al.  Task analysis methods for instructional design , 1999 .

[59]  R. Harden,et al.  Educational strategies in curriculum development: the SPICES model , 1984, Medical education.

[60]  C.L.A. van Herwaarden,et al.  The 2009 framework for undergraduate medical education in the Netherlands , 2010 .

[61]  Olle ten Cate,et al.  Developing Entrustable Professional Activities for Entry Into Clerkship , 2016, Academic medicine : journal of the Association of American Medical Colleges.

[62]  Richard M. Frankel,et al.  Transforming Primary Care Training—Patient-Centered Medical Home Entrustable Professional Activities for Internal Medicine Residents , 2013, Journal of General Internal Medicine.

[63]  Dorene Balmer,et al.  The greater good: how supervising physicians make entrustment decisions in the pediatric emergency department. , 2014, Academic pediatrics.

[64]  Eric S. Holmboe,et al.  Tools for direct observation and assessment of clinical skills of medical trainees: a systematic review. , 2009, JAMA.

[65]  Jennifer Sparks,et al.  Entrustable professional activities in family medicine. , 2013, Journal of graduate medical education.

[66]  Paul Barach,et al.  How do supervising physicians decide to entrust residents with unsupervised tasks? A qualitative analysis. , 2014, Journal of hospital medicine.

[67]  Chen-Lin C. Kulik,et al.  Effectiveness of Mastery Learning Programs: A Meta-Analysis , 1990 .

[68]  C. Vleuten.  Programmatic assessment: From assessment of learning to assessment for learning , 2011 .

[69]  Learning objectives for medical student education--guidelines for medical schools: report I of the Medical School Objectives Project. , 1999, Academic medicine : journal of the Association of American Medical Colleges.

[70]  C. Van herwaarden,et al.  The 2009 framework for undergraduate medical education in the Netherlands , 2010, GMS Zeitschrift fur medizinische Ausbildung.

[71]  Marieke van der Schaaf,et al.  Essential facets of competence that enable trust in graduates: a delphi study among physician educators in the Netherlands. , 2013, Journal of graduate medical education.

[72]  Fedde Scheele,et al.  Viewpoint: Competency-Based Postgraduate Training: Can We Bridge the Gap between Theory and Clinical Practice? , 2007 .

[73]  Tanya Horsley,et al.  Toward a definition of competency-based education in medicine: a systematic review of published definitions , 2010, Medical teacher.

[74]  Sumant R Ranji,et al.  Identifying entrustable professional activities in internal medicine training. , 2013, Journal of graduate medical education.

[75]  J. Norcini,et al.  Workplace-based assessment as an educational tool: AMEE Guide No. 31 , 2007, Medical teacher.

[76]  Mark A Albanese,et al.  Defining characteristics of educational competencies , 2008, Medical education.

[77]  Henry E Fessler,et al.  Entrustable professional activities and curricular milestones for fellowship training in pulmonary and critical care medicine: report of a multisociety working group. , 2014, Chest.

[78]  Carol Carraccio,et al.  Pediatrics redesign project: a pilot implementing competency-based education across the continuum. , 2011, Academic medicine : journal of the Association of American Medical Colleges.

[79]  Georges Bordage,et al.  Supervising incoming first‐year residents: faculty expectations versus residents' experiences , 2014, Medical education.

[80]  Peter Maurer,et al.  The Cambridge Handbook of the Learning Sciences , 2022 .

[81]  Olle ten Cate,et al.  Entrustability of professional activities and competency‐based training , 2005 .

[82]  Marieke van der Schaaf,et al.  Essential facets of competence that enable trust in medical graduates: a ranking study among physician educators in two countries , 2013, Perspectives on Medical Education.

[83]  Robert Englander,et al.  Entrustment Decision Making in Clinical Training , 2016, Academic medicine : journal of the Association of American Medical Colleges.

[84]  Carol Carraccio,et al.  Medical competence: The interplay between individual ability and the health care environment , 2010, Medical teacher.

[85]  Elaine F Dannefer,et al.  The portfolio approach to competency-based assessment at the Cleveland Clinic Lerner College of Medicine. , 2007, Academic medicine : journal of the Association of American Medical Colleges.

[86]  J. H. Davis,et al.  An integrative model of organizational trust, Academy of Management Review, : . , 1995 .

[87]  P. O'Sullivan,et al.  Sequencing learning experiences to engage different level learners in the workplace: An interview study with excellent clinical teachers , 2015, Medical teacher.

[88]  David M. Irby,et al.  Educating Physicians: A Call for Reform of Medical School and Residency , 2010 .

[89]  Jason R. Frank,et al.  The CanMEDS 2005 Physician Competency Framework: Better standards. Better physicians. Better care. , 2005 .

[90]  Stephen Richard Billett,et al.  Mimetic Learning at Work: Learning in the Circumstances of Practice , 2014 .

[91]  T. Whalen,et al.  New Supervision Standards: Discussion and Justification , 2011 .

[92]  B Jolly,et al.  Can I leave the theatre? A key to more reliable workplace-based assessment. , 2014, British journal of anaesthesia.

[93]  P. O'Sullivan,et al.  Understanding trust as an essential element of trainee supervision and learning in the workplace , 2013, Advances in Health Sciences Education.

[94]  S. Lurie,et al.  Measurement of the General Competencies of the Accreditation Council for Graduate Medical Education: A Systematic Review , 2009, Academic medicine : journal of the Association of American Medical Colleges.

[95]  P. W. Teunissen,et al.  Unravelling learning by doing , 2009 .

[96]  Marjan J. B. Govaerts,et al.  Broadening Perspectives on Clinical Performance Assessment: Rethinking the Nature of In-training Assessment , 2007, Advances in health sciences education : theory and practice.

[97]  O. ten Cate,et al.  The case for use of entrustable professional activities in undergraduate medical education. , 2015, Academic medicine : journal of the Association of American Medical Colleges.

[98]  Mark D. Reckase,et al.  Demonstration of Portfolios to Assess Competency of Residents , 2004, Advances in health sciences education : theory and practice.

[99]  John Spencer,et al.  How and what do medical students learn in clerkships? Experience based learning (ExBL) , 2014, Advances in health sciences education : theory and practice.

[100]  V. Marsick.  Learning in the workplace , 1987 .

[101]  Fedde Scheele,et al.  Competency-based postgraduate training: can we bridge the gap between theory and clinical practice? , 2007, Academic medicine : journal of the Association of American Medical Colleges.

[102]  D. Long,et al.  Competency-based medical education: theory to practice , 2010, Medical teacher.

[103]  Gautham Reddy,et al.  Entrustable professional activities for gastroenterology fellowship training , 2014, Gastrointestinal endoscopy.

[104]  D. Hunter,et al.  Qualitative Research: Consensus methods for medical and health services research , 1995 .

[105]  Hendrik Drachsler,et al.  Translating Learning into Numbers: A Generic Framework for Learning Analytics , 2012, J. Educ. Technol. Soc..

[106]  Erik W Driessen,et al.  Portfolios for assessment and learning: AMEE Guide no. 45 , 2009, Medical teacher.

[107]  Michael A. Brooks.  Medical Education and the Tyranny of Competency , 2009, Perspectives in biology and medicine.

[108]  Patricia Cornett,et al.  Developing Entrustable Professional Activities as the Basis for Assessment of Competence in an Internal Medicine Residency: A Feasibility Study , 2013, Journal of General Internal Medicine.

[109]  Nathan M Shumway,et al.  Use of Milestones and Development of Entrustable Professional Activities in 2 Hematology/Oncology Training Programs. , 2015, Journal of graduate medical education.

[110]  Paul Barach,et al.  When Do Supervising Physicians Decide to Entrust Residents With Unsupervised Tasks? , 2010, Academic medicine : journal of the Association of American Medical Colleges.

[111]  Jenny Gavriel.  Cognitive apprenticeship , 2015, Education for primary care : an official publication of the Association of Course Organisers, National Association of GP Tutors, World Organisation of Family Doctors.

[112]  C. V. D. van der Vleuten,et al.  Assessing professional competence : from methods to programmes , 2005 .

[113]  William C McGaghie,et al.  Making July Safer: Simulation-Based Mastery Learning During Intern Boot Camp , 2013, Academic medicine : journal of the Association of American Medical Colleges.

[114]  Olle ten Cate,et al.  Medical Education, Competency‐Based , 2014 .

[115]  A. Collins,et al.  Situated Cognition and the Culture of Learning , 1989 .

[116]  Adam A Rosenberg,et al.  Perspective: Competencies, outcomes, and controversy--linking professional activities to competencies to improve resident education and practice. , 2011, Academic medicine : journal of the Association of American Medical Colleges.

[117]  O. ten Cate,et al.  Trusting graduates to enter residency: what does it take? , 2014, Journal of graduate medical education.

[118]  Olle ten Cate,et al.  Trust, competence, and the supervisor's role in postgraduate training , 2006, BMJ : British Medical Journal.

[119]  J. H. Davis,et al.  An Integrative Model Of Organizational Trust , 1995 .

[120]  R. Stiggins.  Assessment Crisis: The Absence of Assessment for Learning , 2002 .

[121]  Mick O'Keeffe,et al.  Clinical competence in developmental‐behavioural paediatrics: Raising the bar , 2014, Journal of paediatrics and child health.

[122]  Allan S Detsky,et al.  Selecting a specialist: adding evidence to the clinical practice of making referrals. , 2014, JAMA.

[123]  Robert Englander,et al.  ANALYSES/LITERATURE REVIEWS: Evaluating Competence Using a Portfolio: A Literature Review and Web-Based Application to the ACGME Competencies , 2004, Teaching and learning in medicine.

[124]  Susan E. Newman,et al.  Cognitive Apprenticeship: Teaching the Craft of Reading, Writing, and Mathematics. Technical Report No. 403. , 1987 .