Program evaluation models and related theories: AMEE Guide No. 67

This Guide reviews theories of science that have influenced the development of common educational evaluation models. Educators can be more confident when choosing an appropriate evaluation model if they first consider the model's theoretical basis against their program's complexity and their own evaluation needs. Reductionism, system theory, and (most recently) complexity theory have inspired the development of models commonly applied in evaluation studies today. This Guide describes experimental and quasi-experimental models, Kirkpatrick's four-level model, the Logic Model, and the CIPP (Context/Input/Process/Product) model in the context of the theories that influenced their development and that limit or support their ability to do what educators need. The goal of this Guide is for educators to become more competent and confident in being able to design educational program evaluations that support intentional program improvement while adequately documenting or describing the changes and outcomes—intended and unintended—associated with their programs.

[1]  J. Lopreato,et al.  General system theory : foundations, development, applications , 1970 .

[2]  John Goldie,et al.  AMEE Education Guide no. 29: Evaluating educational programmes , 2006, Medical teacher.

[3]  Michael Quinn Patton,et al.  Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use , 2010 .

[4]  Donald L. Kirkpatrick,et al.  Great Ideas Revisited. Techniques for Evaluating Training Programs. Revisiting Kirkpatrick's Four-Level Model. , 1996 .

[5]  D. Musick,et al.  A Conceptual Model for Program Evaluation in Graduate Medical Education , 2006, Academic medicine : journal of the Association of American Medical Colleges.

[6]  M. Poza [ACGME. Accreditation Council for Graduate Medical Education. Recommendations on the schedule for medical residents and their training environment]. , 2002, Neurocirugia.

[7]  Steven J Durning,et al.  The Structure of Program Evaluation: An Approach for Evaluating a Course, Clerkship, or Components of a Residency or Fellowship Training Program , 2007, Teaching and learning in medicine.

[8]  W. Wiersma Evaluation Theory, Models, & Applications , 2008 .

[9]  Elwood F. Holton The flawed four‐level evaluation model , 1996 .

[10]  J. Rabek,et al.  Effects of comprehensive educational reforms on academic success in a diverse student body , 2010, Medical education.

[11]  มาเรียม นิลพันธุ์,et al.  Evaluation Theory, Models, & Applications , 2013 .

[12]  Reid Bates A critical analysis of evaluation practice: the Kirkpatrick model and the principle of beneficence , 2004 .

[13]  Diana H. J. M. Dolmans,et al.  International handbook of research in medical education , 2002 .

[14]  Stewart I. Donaldson,et al.  Program Theory-Driven Evaluation Science: Strategies and Applications , 2007 .

[15]  Eric Allender,et al.  Complexity Theory , 1997, Encyclopedia of Cryptography and Security.

[16]  Functions and structure of a Medical School. , 1974, Connecticut medicine.

[17]  David A Cook,et al.  Twelve tips for evaluating educational programs , 2010, Medical teacher.

[18]  S. Mennin Complexity and health professions education. , 2010, Journal of evaluation in clinical practice.

[19]  Daniela C. Schröter,et al.  Logic Modeling Methods in Program Evaluation , 2008, Journal of MultiDisciplinary Evaluation.

[20]  M. Meehan General System Theory: Foundations, Development, Applications , 1969 .

[21]  Chris L. S. Coryn,et al.  Evaluation Theory, Models, and Applications , 2007 .

[22]  Stewart Mennin,et al.  Complexity and health professions education: a basic glossary. , 2010, Journal of evaluation in clinical practice.

[23]  Tineke A. Abma,et al.  Responsive evaluation: Its meaning and special contribution to health promotion , 2005 .

[24]  Arnold Businger The structure of a program , 1985 .

[25]  K. Hupfeld,et al.  Program evaluation. , 1989, NLN publications.

[26]  W. Doll,et al.  Complexity and the health care professions. , 2010, Journal of evaluation in clinical practice.

[27]  W. Shadish,et al.  Foundations of Program Evaluation: Theories of Practice , 1990 .

[28]  Tom Carney Fourth Generation Evaluation , 1991 .