TECHNIQUES IN EVALUATION

Understanding of any complex process such as evaluation is facilitated by looking at one piece of the total process at a time. Data collection, as one phase of evaluation, is the focus of this chapter. It must, however, be em­ phasized that it is really not possible to discuss anyone phase in isolation. All decisions within the evaluation process are interrelated and dependent on both earlier and subsequent decisions. So while the discussion here focuses on data collection, we need to retain our perspective of it as only one aspect of evaluation, interrelated with others. This chapter is divided into three sections. The first is a discussion of a variety of issues related to data collection, including (1) an analysis of the impact of the evaluator's frame of reference, (2) the relationship of formative and summative evaluation to data collection, and (3) the multiple approaches to deciding what kinds of data should be collected. The second section ex­ amines actual data-collection techniques which could be used in evaluation of continuing education in the health professions. The final section provides critical commentary related to the Standards and data collection.

[1]  O. Laxdal.  Needs assessment in continuing medical education: a practical guide. , 1982, Journal of medical education.

[2]  L. Christman.  The Quality of Medical Care: Evaluation and Improvement , 1977 .

[3]  R. Weiss,et al.  Results of objective evaluation of postgraduate urologic seminar. , 1973, The Journal of urology.

[4]  E. Rubenstein.  Continuing medical education at Stanford: the back-to-medical-school program. , 1973, Journal of Medical Education.

[5]  W. Rosser,et al.  Improving benzodiazepine prescribing in family practice through review and education. , 1981, Canadian Medical Association journal.

[6]  D. Bertram,et al.  The Evaluation of Continuing Medical Education: A Literature Review , 1977, Health education monographs.

[7]  R. Barnes,et al.  EVALUATION OF TEACHING EFFORTS WITH NON-PSYCHIATRIC MEDICAL PRACTITIONERS. , 1964, Diseases of the nervous system.

[8]  J. Singer,et al.  Assessing health quality--the case for tracers. , 1973, The New England journal of medicine.

[9]  E. O. Hirsch.  Utilization review as a means of continuing education. , 1974, Medical care.

[10]  S. C. Driver,et al.  A comparison of three methods of using television for the continuing medical education of general practitioners , 1972, British journal of medical education.

[11]  A. Nelson.  Orphan data and the unclosed loop: a dilemma in PSRO and medical audit. , 1976, The New England journal of medicine.

[12]  D L Sackett,et al.  A randomized trial of continuing medical education. , 1982, The New England journal of medicine.

[13]  C. H. Wood,et al.  AN EXPERIMENT IN POSTGRADUATE EDUCATION TO EVALUATE TEACHING AND EXAMINATION TECHNIQUES. , 1965, Journal of medical education.

[14]  S. Abrahamson,et al.  Effectiveness of continuing medical education: a review of the evidence. , 1979, Evaluation & the health professions.

[15]  J. Ware,et al.  Continuing education, clinical competence, and specialty certification. , 1971, Journal of medical education.

[16]  J. Ware,et al.  Evaluation of weekend seminars for physicians. , 1972, Journal of Medical Education.

[17]  J. Devitt.  Does continuing medical education by peer review really work? , 1973, Canadian Medical Association journal.

[18]  J. Eisenberg,et al.  Computer-based Audit to Detect and Correct Overutilization of Laboratory Tests , 1977, Medical care.

[19]  D. Babbott,et al.  AUSCULTATORY SKILL: GAIN AND RETENTION AFTER INTENSIVE INSTRUCTION. , 1964, Journal of medical education.

[20]  D. Swendeman,et al.  An investigation of behavior change in continuing dental education. , 1976, Journal of dental education.

[21]  J S Cameron.  Broadcast Television for Doctors—a First Evaluation , 1966, British medical journal.

[22]  A. Streissguth,et al.  Change in obstetrician advice following a two-year community educational program on alcohol use and pregnancy. , 1983, American journal of obstetrics and gynecology.

[23]  C R Brown,et al.  Mandatory continuing education. Sense or nonsense? , 1970, JAMA.

[24]  N J Gilman,et al.  Determining educational needs in the physician's office. , 1980, JAMA.

[25]  F. T. de Dombal,et al.  Human and Computer-aided Diagnosis of Abdominal Pain: Further Report with Emphasis on Performance of Clinicians , 1974, British medical journal.

[26]  G. Miller,et al.  Continuing education and patient care research. Physician response to screening test results. , 1967, JAMA.

[27]  A. Zauber,et al.  Drug Utilization Review in an HMO I. Introduction and Examples of Methodology , 1977, Medical care.

[28]  Shepherd Rm.  Evaluation of continuing education. , 1971 .

[29]  T. McClellan,et al.  Description and evaluation of dentist-dental assistant team training in efficient dental practice management. , 1968, Journal of the American Dental Association.

[30]  James R. Sanders,et al.  Educational Evaluation Theory and Practice , 1975 .

[31]  C. B. Petersen.  An Evaluation of the Impact of a Continuing Education Course in Diabetes , 1982, Evaluation & the health professions.

[32]  J. Devitt,et al.  Can patient care audit change doctor performance? , 1975, Journal of medical education.

[33]  P. Woog,et al.  Evaluating Continuing Education , 1980 .

[34]  Williamson Jw.  Health accounting: an outcome-based system of quality assurance: illustrative application to hypertension. , 1975 .

[35]  L. Muhlbaier,et al.  Improving Hypertension Control: Impact of Computer Feedback and Physician Education , 1981, Medical care.

[36]  Stephen B. Soumerai,et al.  Improving Drug-Therapy Decisions through Educational Outreach , 1983 .

[37]  Condon Mb.  Inservice education ... impact on patient care. , 1972 .

[38]  J. Durlak,et al.  Preliminary evaluation of a hospital-based continuing education workshop on death and dying. , 1977, Journal of medical education.

[39]  Cayten Cg,et al.  Surgical audit using predetermined weighted criteria. , 1974 .

[40]  D. Ashbaugh,et al.  Continuing medical education. The philosphy and use of audit. , 1976, Journal of the American Medical Association (JAMA).

[41]  C. Lewis,et al.  Continuing medical education--an epidemiologic evaluation. , 1970, The New England journal of medicine.

[42]  D. Reed,et al.  Continuing education based on record audit in a community hospital. , 1973, Journal of medical education.

[43]  H. Bacal.  Training in Psychological Medicine: An Attempt to Assess Tavistock Clinic Seminars , 1971, Psychiatry in Medicine.

[44]  H. Neu,et al.  Testing the physician's knowledge of antibiotic use: Self-assessment and learning via videotape. , 1975, The New England journal of medicine.