Identifying, Appraising, and Implementing Medical Education Curricula: A Guide for Medical Educators

As physicians' requirements for knowledge and skills evolve, medical educators at all levels often encounter the need for new curricula. Herein, I define curriculum as a planned educational experience that encompasses behavioral goals, instructional methods, and the actual experience of the learners. Educators may wish to develop a new curriculum but find the task daunting because they lack time or training in curriculum science. As a more expedient alternative, they can try to find an established curriculum. While seemingly straightforward, this approach presents its own challenges. Curriculum reports are often not published in indexed journals, not published at all, or not disseminated in any way. Furthermore, physicians involved in medical education may have difficulty judging the quality of curriculum documents. Finally, educators can rarely implement a borrowed curriculum as is, but instead must adapt it to local concerns. Drawing on the work of general curriculum theorists, several investigators have developed approaches to developing new medical education curricula (16). However, the literature on appraising existing curricula remains largely limited to editorial guidelines for reviewing education manuscripts (79) and one analysis of curriculum development reporting (10), despite pleas for scrutiny of the evidence in education (11, 12). I offer a guide for educators who are trying to adapt established curricula to meet their program's needs. I describe several resources for finding medical education curricula. Next, I present an approach to critically appraising a curriculum document and apply this method to a published managed care curriculum. Finally, I demonstrate how this off-the-shelf curriculum can be modified and implemented by another institution. Place yourself in this often-replayed educational conundrum: You are a program director of an internal medicine residency program, and you have just survived a review and site visit by the residency review committee. The Accreditation Council for Graduate Medical Education continued your accreditation, but gave you a shorter review cycle because of a few citations relating to on-call rooms and the lack of a well-documented managed care curriculum. After meeting with a hospital administrator to discuss the former, you turn your attention to the formidable task of developing a managed care curriculum. Before forging ahead on your own, you decide to save yourself time by finding a road tested ready-made curriculum. Your first task is to search for a report of a managed care curriculum, preferably developed for resident education. Identifying Medical Education Curricula Published reports of curricula in many journals can be located by searching the MEDLINE database, which offers breadth and depth of information but suffers from imperfect indexing and a sometimes unmanageable number of references. While not foolproof, the following strategy will improve the efficiency of your search. Combine (by using OR) all of the Medical Subject Heading (MeSH) terms, or text words, related to the medical topic. Then, use AND to combine this list with curriculum (MeSH term) and (AND) one or more of the learner-level MeSH terms listed in Table 1. In addition to inevitably missing some indexed articles, this approach will also miss curricula reported in some international journals not indexed in MEDLINE. Hence, it is useful to manually (or electronically) search the contents of journals specifically devoted to medical education (Table 2). Table 1. Medical Subject Headings for Medical Education Curricula Table 2. Curriculum Resources You should not restrict yourself to published reports, based on the assumption that the best curricula get published. Although negative studies of clinical interventions are less likely to be published, this publication bias may operate differently in education scholarship. Clinician-educators, whose positions do not require national reputations, may not be inclined to report their curricula. Therefore, innovative curricula may not be captured by searching journals. Fortunately, there are other ways to access this information. The U.S. Department of Education and the National Library of Medicine support the Educational Resources Information Center, which contains more than 950 000 abstracts of unpublished documents and journal articles on education research and practice. The Association of Program Directors in Internal Medicine maintains an educational clearinghouse on the Internet, which includes a peer-reviewed annotated list of published and unpublished curricula and related educational literature. Finally, various medical education organizations often disseminate curricula, curricular materials, workshops, guides, or consultations. The Web sites for these organizations are listed in Table 2. In this scenario, a search of these resources captured 18 curriculum documents, including a report of a managed care curriculum for internal medicine residents published in Academic Medicine (13). Gomez and colleagues describe a curriculum in which residents actively participate in utilization review at two primary care practices. Critically Appraising a Curriculum Document Now that you have identified a candidate curriculum, your next task is to determine its quality and suitability for implementation at your institution. In this appraisal, you should consider the curriculum's development phase, learner and program characteristics, learning objectives, instructional strategies, feasibility and sustainability, and effectiveness (Table 3). It is important to keep in mind that any successful curriculum, whether borrowed or new, must respond to local concerns. Thus, your appraisal process has two purposes: to recognize completely incompatible or methodologically unsound curricula and to identify areas for modification, additional information, and additional resources in a curriculum that otherwise meets your needs. Appraisal of published reports may be especially challenging because journals do not follow a uniform format and authors often fail to describe essential elements of the curriculum development process (10). One group has called for uniform reporting and offered an intuitive outline for writing curriculum development articles (14). Table 3. Critical Appraisal Guide for Medical Education Curricula Curriculum Development Several steps in the development phase inform the selection of curriculum content and design of educational strategies. Most important, needs analysis ensures that the curriculum responds to both societal mandates and institutional concerns. To establish general needs, planners should argue that a new approach to education is needed, citing deficiencies of existing curricula and evidence that practicing physicians do not exhibit the expected skills, behaviors, or attitudes. Local needs assessment targets learners at a particular program. This can be accomplished by using several methods in combination, including surveys, focus groups, faculty interviews, or formal assessments of the learners. Additional development elements include review of other curricula and related education literature; cataloguing of available resources; and final decision making about major issues in content, structure, and instructional methods. Planners should describe these development steps in sufficient detail to allow others to replicate the process as they adapt the curriculum for their learners. In their report of a managed care curriculum, Gomez and colleagues (13) effectively made the case for innovations in managed care training, arguing that while few physicians-in-training can expect to avoid practicing in a managed care environment, many feel poorly trained to accommodate this increased scrutiny of health and fiscal outcomes. This deficiency may be particularly glaring in public hospitals or academic centers, where managed care experiences are not common and are not mandated. However, Gomez and colleagues did not report an analysis of their program's local needs. Learner and Program Characteristics Information about the learners' demographic characteristics and the training program's characteristics will help you assess the external validity (or generalizability) of the curriculum. In particular, do their learners resemble your learners closely enough that they can expect a similar experience and effect of the curriculum? However, because you will probably modify any curriculum, you should not necessarily reject a curriculum for an inexact match, unless the differences are so great that you cannot extrapolate it to your learners. This managed care curriculum was developed for two internal medicine residency programs, including categorical, preliminary, and primary care residents. The curriculum was implemented at a university hospitalbased office practice with high managed care penetration and financial incentives to limit resource utilization and a Veterans Administration hospitalbased clinic with no requirements to control subspecialty referrals. In this report, the authors did not describe the learners. However, in separate publications (15, 16), which include more details about their evaluation study, they reported that 50% of the residents were women and that the residents were equally divided among postgraduate training years. Learning Goals and Objectives A curriculum's broad aim, often called a mission statement, offers the reader an overall sense of the planners' educational intent and instructional strategies. You may decide to move on without further appraisal if the mission statement seems incongruous with your needs. Although all planners are guided, at least implicitly, by certain educational goals, some explicitly translate these goals into more concrete learning objectives. This approach offers several advantages, including exposing gaps in current experiences; clearly communicating the curriculum intent; helping learners organiz