Evidence for curricular and instructional design approaches in undergraduate medical education: An umbrella review

Abstract Introduction: An umbrella review compiles evidence from multiple reviews into a single accessible document. This umbrella review synthesizes evidence from systematic reviews on curricular and instructional design approaches in undergraduate medical education, focusing on learning outcomes. Methods: We conducted bibliographic database searches in Medline, EMBASE and ERIC from database inception to May 2013 inclusive, and digital keyword searches of leading medical education journals. We identified 18,470 abstracts; 467 underwent duplicate full-text scrutiny. Results: Thirty-six articles met all eligibility criteria. Articles were abstracted independently by three authors, using a modified Kirkpatrick model for evaluating learning outcomes. Evidence for the effectiveness of diverse educational approaches is reported. Discussion: This review maps out empirical knowledge on the efficacy of a broad range of educational approaches in medical education. Critical knowledge gaps, and lapses in methodological rigour, are discussed, providing valuable insight for future research. The findings call attention to the need for adopting evaluative strategies that explore how contextual variabilities and individual (teacher/learner) differences influence efficacy of educational interventions. Additionally, the results underscore that extant empirical evidence does not always provide unequivocal answers about what approaches are most effective. Educators should incorporate best available empirical knowledge with experiential and contextual knowledge.

[1]  David Moher,et al.  Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews , 2007, BMC medical research methodology.

[2]  M. Wilkes,et al.  Doctoring III: innovations in education in the clinical years. , 1995, Academic medicine : journal of the Association of American Medical Colleges.

[3]  Jill Thistlethwaite,et al.  Active patient involvement in the education of health professionals , 2010, Medical education.

[4]  Greg Ogrinc,et al.  Realist evaluation as a framework for the assessment of teaching about the improvement of care. , 2009, The Journal of nursing education.

[5]  C. Webb,et al.  Portfolios and assessment of competence: a review of the literature. , 2003, Journal of advanced nursing.

[6]  A. Kolmos,et al.  Problem Based Learning , 2007 .

[7]  S. Reeves,et al.  A best evidence systematic review of interprofessional education: BEME Guide no. 9 , 2007, Medical teacher.

[8]  A. Oxman,et al.  Overviews of Reviews , 2008 .

[9]  Prathibha Varkey,et al.  The effectiveness of self‐directed learning in health professions education: a systematic review , 2010, Medical education.

[10]  N. Mattheos,et al.  Problem-based learning in academic health education. A systematic literature review. , 2010, European journal of dental education : official journal of the Association for Dental Education in Europe.

[11]  T. Roberts,et al.  Strategies and interventions for the involvement of real patients in medical education: a systematic review , 2009, Medical education.

[12]  David A Cook,et al.  Time and learning efficiency in Internet-based learning: a systematic review and meta-analysis , 2010, Advances in health sciences education : theory and practice.

[13]  William C. McGaghie,et al.  Simulations in Professional Competence Assessment: Basic Considerations , 1999 .

[14]  T S Ikonen,et al.  Virtual Reality Simulator Training of Laparoscopic Cholecystectomies — A Systematic Review , 2012, Scandinavian journal of surgery : SJS : official organ for the Finnish Surgical Society and the Scandinavian Surgical Society.

[15]  A. Levinson.  Where is evidence‐based instructional design in medical education curriculum development? , 2010, Medical education.

[16]  S. B. Bierer,et al.  How to Measure Success: The Impact of Scholarly Concentrations on Students—A Literature Review , 2010, Academic medicine : journal of the Association of American Medical Colleges.

[17]  Shai Ben-David,et al.  On self-directed learning , 1995, COLT '95.

[18]  J. Michael Spector,et al.  Handbook of Research on Educational Communications and Technology, 3rd Edition , 2012 .

[19]  J. Thistlethwaite,et al.  The effectiveness of case-based learning in health professional education. A BEME systematic review: BEME Guide No. 23 , 2012, Medical teacher.

[20]  Bierer Sb,et al.  How to measure success: the impact of scholarly concentrations on students--a literature review. , 2010 .

[21]  Lester R. Bittel,et al.  TRAINING AND DEVELOPMENT HANDBOOK. , 1967 .

[22]  Heidi S Chumley-Jones,et al.  Web‐based Learning: Sound Educational Method or Hype? A Review of the Evaluation Literature , 2002, Academic medicine : journal of the Association of American Medical Colleges.

[23]  N. Kerse,et al.  Effectiveness of simulated clinical teaching in general practice: randomised controlled trial. , 2012, Journal of primary health care.

[24]  C. Adams,et al.  Educational games for mental health professionals: a Cochrane review. , 2007, The international journal of psychiatric nursing research.

[25]  C. R. Larsen,et al.  The efficacy of virtual reality simulation training in laparoscopy: a systematic review of randomized trials , 2012, Acta obstetricia et gynecologica Scandinavica.

[26]  Lisa Hartling,et al.  The effects of audience response systems on learning outcomes in health professions education. A BEME systematic review: BEME Guide No. 21 , 2012, Medical teacher.

[27]  N. Fernandez,et al.  Evidence‐based arguments in support of medical education reform , 2014, Medical education.

[28]  T. Dornan,et al.  What has changed in the evidence for early experience? Update of a BEME systematic review , 2010, Medical teacher.

[29]  Tyrone Donnon,et al.  The effect of simulation in improving students’ performance in laparoscopic surgery: a meta-analysis , 2012, Surgical Endoscopy.

[30]  Rebecca A Dumont,et al.  Effects of international health electives on medical student learning and career choice: results of a systematic literature review. , 2011, Family medicine.

[31]  Win May,et al.  A ten-year review of the literature on the use of standardized patients in teaching and learning: 1996–2005 , 2009, Medical teacher.

[32]  S. Reeves,et al.  A Critical Review of Evaluations of Interprofessional Education , 2002 .

[33]  C. Moyer,et al.  Expanding the Boundaries of Medical Education: Evidence for Cross-Cultural Exchanges , 2003, Academic medicine : journal of the Association of American Medical Colleges.

[34]  Maurice Mars,et al.  A systematic review of the effectiveness of videoconference-based tele-education for medical and nursing education. , 2012, Worldviews on evidence-based nursing.

[35]  O. Hyder,et al.  How has problem based learning fared in Pakistan? , 2012, Journal of the College of Physicians and Surgeons--Pakistan : JCPSP.

[36]  R. Hatala,et al.  Simulation-Based Training for Cardiac Auscultation Skills: Systematic Review and Meta-Analysis , 2013, Journal of General Internal Medicine.

[37]  J Spencer,et al.  How can experience in clinical and community settings contribute to early medical education? A BEME systematic review , 2006, Medical teacher.

[38]  H S Barrows,et al.  A taxonomy of problem‐based learning methods , 1986, Medical education.

[39]  Michele Tarsilla.  Cochrane Handbook for Systematic Reviews of Interventions , 2010, Journal of MultiDisciplinary Evaluation.

[40]  R. Haynes,et al.  Optimal search strategies for retrieving systematic reviews from Medline: analytical survey , 2004, BMJ : British Medical Journal.

[41]  Frazier T. Stevenson,et al.  Comparing Problem-Based Learning with Case-Based Learning: Effects of a Major Curricular Shift at Two Institutions , 2007, Academic medicine : journal of the Association of American Medical Colleges.

[42]  C. Popovic,et al.  The educational effects of portfolios on undergraduate student learning: A Best Evidence Medical Education (BEME) systematic review. BEME Guide No. 11 , 2009, Medical teacher.

[43]  Jeremy M. Grimshaw,et al.  Increasing the demand for childhood vaccination in developing countries: a systematic review , 2009, BMC international health and human rights.

[44]  S. Reeves,et al.  Affordances of Knowledge Translation in Medical Education: A Qualitative Exploration of Empirical Knowledge Use Among Medical Educators , 2015, Academic medicine : journal of the Association of American Medical Colleges.

[45]  D. Altman.  Doctoring III: Innovations in Education in the Clinical Years. , 1995 .

[46]  H. Yuan,et al.  A systematic review of selected evidence on improving knowledge and skills through high-fidelity simulation. , 2012, Nurse education today.

[47]  Lisa Hartling,et al.  The Effects of Audience Response Systems on Learning Outcomes in Health Professions Education , 2011 .

[48]  B. Daley,et al.  Concept maps in medical education: an analytical literature review , 2010, Medical education.

[49]  P. Baker,et al.  Framework for action on interprofessional education and collaborative practice , 2010 .

[50]  Maria J Grant,et al.  A typology of reviews: an analysis of 14 review types and associated methodologies. , 2009, Health information and libraries journal.

[51]  J. Colliver.  Effectiveness of Problem‐based Learning Curricula: Research and Theory , 2000, Academic medicine : journal of the Association of American Medical Colleges.

[52]  Wang Qing.  A Ten-year Review of the Literature on the Use of Standardized Patients in Teaching and Learning , 2009 .

[53]  Diana K. Kelly,et al.  Developing the role of patients as teachers: literature review , 2002, BMJ : British Medical Journal.

[54]  T. Spencer,et al.  Evidence-based Practice: A Framework for Making Effective Decisions , 2012 .

[55]  M. Silberman,et al.  The handbook of experiential learning , 2007 .

[56]  Aleksandra Cislak,et al.  Effects of School-based Interventions Targeting Obesity-Related Behaviors and Body Weight Change: A Systematic Umbrella Review , 2011, Behavioral medicine.

[57]  Training,et al.  International handbook of education for the changing world of work : bridging academic and vocational learning , 2009 .

[58]  J. Frank,et al.  Développer la capacité d’application des connaissances en santé publique , 2007, Canadian Journal of Public Health = Revue Canadienne de Santé Publique.

[59]  T. Levett-Jones,et al.  A systematic review of the effectiveness of interprofessional education in health professional programs. , 2013, Nurse education today.

[60]  J. Ioannidis,et al.  Vitamin D and multiple health outcomes: umbrella review of systematic reviews and meta-analyses of observational studies and randomised trials , 2014, BMJ : British Medical Journal.

[61]  L. Sjodahl.  Evaluation of Training. , 1975 .

[62]  D. T. Vernon,et al.  Does problem‐based learning work? A meta‐analysis of evaluative research , 1993, Academic medicine : journal of the Association of American Medical Colleges.

[63]  Val Wass,et al.  Portfolios in medical education: why do they meet with mixed success? A systematic review , 2007, Medical education.

[64]  B. N. Harder,et al.  Use of simulation in teaching and learning in health sciences: a systematic review. , 2010, The Journal of nursing education.

[65]  K. Walters,et al.  Improving teaching about medically unexplained symptoms for newly qualified doctors in the UK: findings from a questionnaire survey and expert workshop , 2017, BMJ Open.

[66]  C E Engel,et al.  Educational technology in medical education. , 1972, Proceedings of the Royal Society of Medicine.

[67]  Martha Reeves,et al.  Evaluation of Training , 1993 .

[68]  D. Allen,et al.  What is Problem-Based Learning? , 2006 .

[69]  H. Ageely,et al.  The role of traditional dissection in medical education , 2010 .

[70]  Lisa Hartling,et al.  Problem-based learning in pre-clinical medical education: 22 years of outcome research , 2010, Medical teacher.

[71]  E. Di Ruggiero,et al.  Developing knowledge translation capacity in public health: the role of the National Collaborating Centres. , 2007, Canadian journal of public health = Revue canadienne de sante publique.

[72]  D. Cook,et al.  Computerized Virtual Patients in Health Professions Education: A Systematic Review and Meta-Analysis , 2010, Academic medicine : journal of the Association of American Medical Colleges.

[73]  McGaghie Wc,et al.  Innovative Simulations for Assessing Professional Competence , 1999 .

[74]  M. Petticrew,et al.  Working for health? Evidence from systematic reviews on the effects on health and health inequalities of organisational changes to the psychosocial work environment. , 2009, Preventive medicine.

[75]  Stephen Brookfield,et al.  Self-Directed Learning , 2009 .

[76]  Simon Cooper,et al.  Educational gaming in the health sciences: systematic review. , 2009, Journal of advanced nursing.

[77]  T. Kind,et al.  Characteristics of multi-institutional health sciences education research: a systematic review , 2017, Journal of the Medical Library Association : JMLA.

[78]  A. Winkelmann,et al.  Anatomical dissection as a teaching method in medical school: a review of the evidence , 2007, Medical education.

[79]  Vivienne Bozalek,et al.  The role of blended learning in the clinical education of healthcare students: A systematic review , 2012, Medical teacher.

[80]  Fabrizio Consorti,et al.  Efficacy of virtual patients in medical education: A meta-analysis of randomized studies , 2012, Comput. Educ..

[81]  Dario M Torre,et al.  Twelve tips for teaching with concept maps in medical education , 2013, Medical teacher.

[82]  R Brian Haynes,et al.  EMBASE search strategies achieved high sensitivity and specificity for retrieving methodologically sound systematic reviews. , 2007, Journal of clinical epidemiology.

[83]  R. Mayer.  Applying the science of learning to medical education , 2010, Medical education.

[84]  Alain Abran,et al.  A systematic literature review: Opinion mining studies from mobile app store user reviews , 2017, J. Syst. Softw..

[85]  G. Gartlehner,et al.  Differences in Efficacy and Safety of Pharmaceutical Treatments between Men and Women: An Umbrella Review , 2010, PloS one.