Towards effective evaluation and reform in medical education: a cognitive and learning sciences perspective

Health professions education is dealing with major transformations in light of the changing nature of the health care delivery system, including the use of technology for “just in time” delivery of care, evidence-based practice, personalized medical care and learning, as health professionals strive to integrate biomedical advances and clinical practice. This has forced the medical education community to reassess the current teaching and learning practices and more importantly, the evaluation of the medical education process. There have been recent advances in cognitive and learning sciences theories, some of which can inform medical educators about best teaching and learning practices and their impact on the evaluation process. An understanding of these theories provides a sound rationale for choosing specific instructional strategies and choosing evaluation measures that assess the curricular objectives. The review begins with an overview of evaluation and assessment in education, followed by an overview of major theories from the cognitive and learning sciences. Next, the role of cognitive and learning sciences theories in informing the process of medical education evaluation is discussed, including its impact on student learning, performance and professional competence, as well as recommendations for reform of medical curricula based on such theories. The paper continues with the elaboration of current trends in health sciences education, particularly medical education, and available evidence for the impact on student learning and performance as well as areas where more research is needed.

[1]  B. Bloom Taxonomy of educational objectives , 1956 .

[2]  Robert J. Mislevy,et al.  Test Theory for A New Generation of Tests , 1994 .

[3]  Sanjeev Dutta,et al.  To simulate or not to simulate: what is the question? , 2006, Annals of surgery.

[4]  M. Friedrich,et al.  Practice Makes Perfect , 2002 .

[5]  S. Saha,et al.  Losing Culture on the Way to Competence: The Use and Misuse of Culture in Medical Education , 2006, Academic medicine : journal of the Association of American Medical Colleges.

[6]  R. Sawyer The Cambridge Handbook of the Learning Sciences: Introduction , 2014 .

[7]  D. Perkins,et al.  Rocky Roads to Transfer: Rethinking Mechanism of a Neglected Phenomenon , 1989 .

[8]  R. Steadman,et al.  Simulation-based training is superior to problem-based learning for the acquisition of critical assessment and management skills* , 2006, Critical care medicine.

[9]  D. Gaba The future vision of simulation in health care , 2004, Quality and Safety in Health Care.

[10]  C. Pellegrini To Err Is Human … Or Is It? , 2006 .

[11]  V L Patel,et al.  Cognition and expertise: acquisition of medical competence. , 2000, Clinical and investigative medicine. Medecine clinique et experimentale.

[12]  M. Tervalon Components of Culture in Health for Medical Students' Education , 2003, Academic medicine : journal of the Association of American Medical Colleges.

[13]  Benjamin S. Bloom,et al.  A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives , 2000 .

[14]  G. Regehr,et al.  The life long challenge of expertise. , 2001, Medical education.

[15]  A. Gallagher,et al.  Approval of virtual reality training for carotid stenting: what this means for procedural-based medicine. , 2004, JAMA.

[16]  R. Sternberg,et al.  Transfer on Trial: Intelligence, Cognition and Instruction , 1993 .

[17]  E. Baker Moving to the Next Generation System Design: Integrating Cognition, Assessment, and Learning. CSE Report 706. , 2007 .

[18]  Jon Veloski,et al.  Systematic review of the literature on assessment, feedback and physicians’ clinical performance: BEME Guide No. 7 , 2006, Medical teacher.

[19]  Madhabi Chatterji,et al.  Evidence on “What Works”: An Argument for Extended-Term Mixed-Method (ETMM) Evaluation Designs , 2004 .

[20]  J. Vozenilek,et al.  Addressing the systems-based practice core competency: a simulation-based curriculum. , 2005, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[21]  Danette W. McKinley,et al.  Assessment methods in medical education. , 2007, International journal of health sciences.

[22]  Hossam Hamdy,et al.  BEME systematic review: Predictive values of measurements obtained in medical schools and future performance in medical practice , 2006, Medical teacher.

[23]  J. Beaubien,et al.  Medical Team Training Programs in Health Care , 2005 .

[24]  Elizabeth G Armstrong,et al.  Can a Simulated Critical Care Encounter Accelerate Basic Science Learning Among Preclinical Medical Students? A Pilot Study , 2006, Simulation in healthcare : journal of the Society for Simulation in Healthcare.

[25]  J. Greeno THE SITUATIVITY OF KNOWING, LEARNING, AND RESEARCH , 1998 .

[26]  F. Paas,et al.  Cognitive Architecture and Instructional Design , 1998 .

[27]  J. Shumway,et al.  AMEE Guide No. 25: The assessment of learning outcomes for the competent and reflective physician , 2003, Medical teacher.

[28]  D. Kaufman,et al.  Applying educational theory in practice , 2003, BMJ : British Medical Journal.

[29]  D. Krathwohl A Revision of Bloom's Taxonomy: An Overview , 2002 .

[30]  Kevin W Eva,et al.  How Can I Know What I Don't Know? Poor Self Assessment in a Well-Defined Domain , 2004, Advances in health sciences education : theory and practice.

[31]  C. V. D. Vleuten,et al.  The assessment of professional competence: Developments, research and practical implications , 1996 .

[32]  H. Simon,et al.  Situated Learning and Education1 , 1996 .

[33]  John R. Anderson,et al.  Rules of the Mind , 1993 .

[34]  V. Patel,et al.  Reasoning and Instruction in Medical Curricula , 1993 .

[35]  D. Irby,et al.  Assessment in medical education. , 2007, The New England journal of medicine.

[36]  Paul J. Feltovich,et al.  The Cambridge handbook of expertise and expert performance , 2006 .

[37]  D. Olson,et al.  The Handbook of education and human development : new models of learning, teaching, and schooling , 1996 .

[38]  C. Griffith,et al.  Relationships of how well attending physicians teach to their students' performances and residency choices , 1997, Academic medicine : journal of the Association of American Medical Colleges.

[39]  R. Satava,et al.  Virtual Reality Training Improves Operating Room Performance: Results of a Randomized, Double-Blinded Study , 2002, Annals of surgery.

[40]  Ann L. Brown,et al.  How people learn: Brain, mind, experience, and school. , 1999 .

[41]  C. Hobgood,et al.  Teaching culturally appropriate care: a review of educational models and methods. , 2006, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[42]  K. Holyoak,et al.  The Cambridge handbook of thinking and reasoning , 2005 .

[43]  Daniel L. Schwartz,et al.  Rethinking transfer: A simple proposal with multiple implica-tions , 1999 .

[44]  C. Watkins,et al.  Developing an evidence base for interdisciplinary learning: a systematic review. , 2001, Journal of advanced nursing.

[45]  J. Norcini,et al.  Certifying examination performance and patient outcomes following acute myocardial infarction , 2002, Medical education.

[46]  John R. Anderson,et al.  Implications of the ACT-R Learning Theory: No Magic Bullets , 2000 .

[47]  David Andreoff Evans,et al.  Cognitive Science in Medicine: Biomedical Modeling , 1989 .

[48]  K. Fish,et al.  Testing Internal Consistency and Construct Validity During Evaluation of Performance in a Patient Simulator , 1998, Anesthesia and analgesia.

[49]  A. Ziv,et al.  Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review , 2005, Medical teacher.

[50]  T. Tang,et al.  Implementation and Evaluation of an Undergraduate Sociocultural Medicine Program , 2002, Academic medicine : journal of the Association of American Medical Colleges.

[51]  H. Mcdonald,et al.  Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. , 2005, JAMA.

[52]  L. Snell,et al.  A review of the evaluation of clinical teaching: new perspectives and challenges * , 2000, Medical education.

[53]  K. Holyoak,et al.  Schema induction and analogical transfer , 1983, Cognitive Psychology.

[54]  David P. Sklar,et al.  Human simulation in emergency medicine training: A model curriculum , 2002 .

[55]  Rosemarie Fernandez,et al.  The use of simulation in emergency medicine: a research agenda. , 2007, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[56]  James A. Gordon,et al.  A Randomized Controlled Trial of Simulation-Based Teaching versus Traditional Instruction in Medicine: A Pilot Study among Clinical Medical Students , 2006, Advances in health sciences education : theory and practice.

[57]  Lisbet Meurling,et al.  Target‐focused medical emergency team training using a human patient simulator: effects on behaviour and attitude , 2007, Medical education.

[58]  J. Bransford How people learn , 2000 .

[59]  John C. Morey,et al.  Simulation based teamwork training for emergency department staff: does it improve clinical team performance when added to an existing didactic teamwork curriculum? , 2004, Quality and Safety in Health Care.

[60]  Geoff Norman,et al.  Editorial – Beyond PBL , 2004, Advances in health sciences education : theory and practice.

[61]  C. Nikendei,et al.  Effects of basic clinical skills training on objective structured clinical examination performance , 2005, Medical education.

[62]  J. Betancourt Cross‐cultural Medical Education: Conceptual Approaches and Frameworks for Evaluation , 2003, Academic medicine : journal of the Association of American Medical Colleges.

[63]  The Influence of Testing Context and Clinical Rotation Order on Students’ OSCE Performance , 2004, Academic medicine : journal of the Association of American Medical Colleges.

[64]  W. Bond,et al.  The Status of Human Simulation Training in Emergency Medicine Residency Programs , 2006, Simulation in healthcare : journal of the Society for Simulation in Healthcare.

[65]  A. Sfard,et al.  Telling Identities: In Search of an Analytic Tool for Investigating Learning as a Culturally Shaped Activity , 2005 .

[66]  S. Hawkes,et al.  Diverse realities: sexually transmitted infections and HIV in India , 2002, Sexually transmitted infections.

[67]  Lucy Suchman Plans and situated actions: the problem of human-machine communication , 1987 .

[68]  D. Gentner,et al.  Learning and Transfer: A General Role for Analogical Encoding , 2003 .

[69]  Daniel L. Schwartz,et al.  Chapter 3: Rethinking Transfer: A Simple Proposal With Multiple Implications , 1999 .

[70]  Ok-choon Park,et al.  Hypermedia: functional features and research issues , 1991 .

[71]  G. Hitman,et al.  Relationships between students' clinical experiences in introductory clinical courses and their performances on an objective structured clinical examination (OSCE) , 1996, Academic medicine : journal of the Association of American Medical Colleges.

[72]  G. Norman,et al.  A comparison of behaviour on simulated patients and patient management problems , 1981, Medical education.

[73]  J. Lave Cognition in Practice: Outdoors: a social anthropology of cognition in practice , 1988 .

[74]  G R Norman,et al.  Exploring the etiology of content specificity: factors influencing analogic transfer and problem solving. , 1998, Academic medicine : journal of the Association of American Medical Colleges.

[75]  V. Patel,et al.  Return to basic sciences after clinical experience in undergraduate medical training , 1984, Medical education.

[76]  John R. Anderson ACT: A simple theory of complex cognition. , 1996 .

[77]  Carlos A Pellegrini,et al.  Surgical Education in the United States: Navigating the White Waters , 2006, Annals of surgery.

[78]  Vimla L. Patel,et al.  Cognitive psychological studies of representation and use of clinical practice guidelines , 2001, Int. J. Medical Informatics.

[79]  N. Saquib,et al.  STD in Bangladesh's trucking industry: prevalence and risk factors , 2002, Sexually transmitted infections.

[80]  H. Schmidt,et al.  A Cognitive Perspective on Medical Expertise: Theory and Implications , 1990 .

[81]  Vimla L. Patel,et al.  Causal Explanation of Complex Physiological Concepts by Medical Students. , 1991 .

[82]  R. Mayer,et al.  Cognitive constraints on multimedia learning: When presenting more material results in less understanding. , 2001 .

[83]  Friedrich Mj,et al.  Practice makes perfect: risk-free medical training with patient simulators. , 2002 .

[84]  C. Griffith,et al.  Medical Students And Housestaff: Partners In Learning , 1998 .

[85]  G. Fried,et al.  Proving the Value of Simulation in Laparoscopic Surgery , 2004, Annals of surgery.

[86]  Doreen Cleave-Hogg,et al.  A worldwide survey of the use of simulation in anesthesia , 2002, Canadian journal of anaesthesia = Journal canadien d'anesthesie.

[87]  John R. Anderson The Architecture of Cognition , 1983 .

[88]  Paul J. Feltovich,et al.  Learning, Teaching and Testing for Complex Conceptual Understanding. , 1991 .

[89]  S. Hamstra,et al.  Teaching technical skills to surgical residents: a survey of empirical research. , 2006, Clinical orthopaedics and related research.

[90]  Lisa D Howley,et al.  Performance Assessment in Medical Education , 2004, Evaluation & the health professions.

[91]  Patricia A. Kapur,et al.  Patient Simulator Competency Testing: Ready for Takeoff? , 1998 .

[92]  K. A. Ericsson,et al.  Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. , 2004, Academic medicine : journal of the Association of American Medical Colleges.

[93]  G. Maddern,et al.  Surgical Simulation: A Systematic Review , 2006, Annals of surgery.

[94]  R. Mayer,et al.  Nine Ways to Reduce Cognitive Load in Multimedia Learning , 2003 .