Written case simulations: do they predict physicians' behavior?

Written case simulations are often used to investigate physicians' decision making and clinical competence. Their use rests on the assumption that physicians' responses to written simulations closely agree with their responses to actual clinical encounters, yet this assumption of criterion validity has received little attention. To determine the ability of written case simulations to predict actual clinical behavior, we applied methodologic criteria to published articles that used written simulations. Only 11 (15%) of 74 articles included an assessment of the criterion validity of their written case simulations. Only 2 of those 11 studies were designed and executed in such a way that criterion validity could be fully interpreted. No clear consensus emerged from an examination on the 11 studies on how well responses to written case simulations perform as proxy measures of actual behavior. More work is needed before assuming that written case simulations measure actual behavior.

[1]  J. Gillis,et al.  Drug Therapy Decisions: A Social Judgment Analysis , 1981, The Journal of nervous and mental disease.

[2]  R S Wigton,et al.  Modeling decisions to use tube feeding in seriously ill patients. , 1987, Archives of internal medicine.

[3]  J. Howie.  Further Observations on Diagnosis and Management of General Practice Respiratory Illness using Simulated Patient Consultations , 1974, British medical journal.

[4]  L E Hollister,et al.  Decisions about drug therapy. II. Expert opinion in a hypothetical situation. , 1972, Behavioral science.

[5]  D. Sackett.  Bias in analytic research. , 1979, Journal of chronic diseases.

[6]  R. Pearlman,et al.  Variability in physician estimates of survival for acute respiratory failure in chronic obstructive pulmonary disease. , 1987, Chest.

[7]  M. Rothert.  Physicians' and Patients' Judgments of Compliance with a Hypertensive Regimen , 1982, Medical decision making : an international journal of the Society for Medical Decision Making.

[8]  D. Rovner,et al.  Methods of Analyzing Physician Practice Patterns in Hypertension , 1989, Medical care.

[9]  E. Vayda,et al.  Use of hypothetical cases to investigate indications for surgery. , 1981, Canadian journal of surgery. Journal canadien de chirurgie.

[10]  D. Newble,et al.  Patient management problems Issues of validity , 1982, Medical education.

[11]  B. Pfefferbaum,et al.  Decisions about drug treatment in children. , 1984, Journal of the American Academy of Child Psychiatry.

[12]  N. Hadler,et al.  Medical disability assessment of the back pain patient for the Social Security Administration: the weighting of presenting clinical features. , 1988, Journal of clinical epidemiology.

[13]  D. Hemenway,et al.  Testing for Physician-induced Demand With Hypothetical Cases , 1985, Medical care.

[14]  G. Page,et al.  Performance on PMPs and performance in practice: are they related? , 1980, Journal of medical education.

[15]  W. Carter,et al.  Variability in physician bioethical decision-making. A case study of euthanasia. , 1982, Annals of internal medicine.

[16]  M. J. Long,et al.  The Effects of Price Information on Physicians' Test-Ordering Behavior: Ordering of Diagnostic Tests , 1982, Medical care.

[17]  R. Wachter,et al.  Attitudes of medical residents regarding intensive care for patients with the acquired immunodeficiency syndrome. , 1988, Archives of internal medicine.

[18]  K. Rudestam,et al.  The clinical judgement process in the prescribing of psychotropic drugs. , 1981, The International journal of the addictions.

[19]  K. R. Hammond,et al.  An Experimental Study of the Clinical Judgment of General Physicians in Evaluating and Prescribing for Depression , 1981, British Journal of Psychiatry.

[20]  Calvin G. Barnes,et al.  Judging "current disease activity" in rheumatoid arthritis--an international comparison. , 1983, The Journal of rheumatology.

[21]  V L Patel,et al.  Processing of critical information by physicians and medical students. , 1987, Journal of medical education.

[22]  P. Slovic,et al.  Analyzing use of diagnostic signs. , 1971, Investigative radiology.

[23]  W. McCarthy.  An assessment of the influence of cueing items in objective examinations. , 1966, Journal of Medical Education.

[24]  M. Rothert,et al.  Diagnostic cues in gastroenterology. , 1983, Australian and New Zealand journal of medicine.

[25]  G. Webster,et al.  Reliability, validity and efficiency of multiple choice question and patient management problem item formats in assessment of clinical competence , 1985, Medical education.

[26]  R. Deber.  The Determinants of Treatment Choice in End-stage Renal Disease , 1986, Medical decision making : an international journal of the Society for Medical Decision Making.

[27]  H. I. Schwartz,et al.  Determining resuscitation status: a survey of medical professionals. , 1986, General hospital psychiatry.

[28]  M. Kuyvenhoven,et al.  Written simulation of patient-doctor encounters. 3. Comparison of the performance in the simulation with prescription and referral data in reality. , 1984, Family practice.

[29]  A. Tversky,et al.  The framing of decisions and the psychology of choice. , 1981, Science.

[30]  I. Ajzen.  Attitudes, Traits, and Actions: Dispositional Prediction of Behavior in Personality and Social Psychology , 1987 .

[31]  M. Young,et al.  Do cardiologists have higher thresholds for recommending coronary arteriography than family physicians? , 1987, Health services research.

[32]  W O Spitzer,et al.  Physician Sensitivity to the Price of Diagnostic Tests: A U.S.—Canadian Analysis , 1982, Medical care.

[33]  C. Cassel,et al.  Nurses' and Physicians' Attitudes Toward Tube‐Feeding Decisions in Long‐Term Care , 1986, Journal of the American Geriatrics Society.

[34]  M. Rothert,et al.  Physicians' Judgments About Estrogen Replacement Therapy for Menopausal Women , 1984, Obstetrics and gynecology.

[35]  E. Vayda,et al.  Measuring surgical decision-making with hypothetical cases. , 1982, Canadian Medical Association journal.

[36]  S. Rachman,et al.  The utility of analogue research. , 1979, Behaviour research and therapy.

[37]  K. Hawton,et al.  The attitudes of psychiatrists to deliberate self-poisoning: comparison with physicians and nurses. , 1981, The British journal of medical psychology.

[38]  F. L. Newman,et al.  Constructing case vignettes for evaluating clinical judgment: An empirical model , 1984 .

[39]  J. Feussner,et al.  Blood alcohol measurements in the emergency department: who needs them? , 1988, American journal of public health.

[40]  D M Chaput de Saintonge,et al.  Clinical judgment in rheumatoid arthritis. I. Rheumatologists' opinions and the development of 'paper patients'. , 1983, Annals of the rheumatic diseases.

[41]  J S Gonnella,et al.  The validity of Patient Management Problems. , 1973, Journal of medical education.

[42]  D. Campbell,et al.  Convergent and discriminant validation by the multitrait-multimethod matrix. , 1959, Psychological bulletin.

[43]  D. Rose,et al.  A model for physicians' therapeutic decision making. , 1987, Archives of internal medicine.

[44]  A. Gittelsohn,et al.  Surgical Decision Making: The Reliability of Clinical Judgment , 1979, Annals of surgery.

[45]  S. Perry,et al.  Management of pain during debridement: a survey of U.S. burn units , 1982, PAIN.

[46]  G. Longstreth.  Detecting colorectal neoplasms. Assessment based on hypothetical cases. , 1987, Journal of clinical gastroenterology.

[47]  R. Kane,et al.  Physicians' Attitudes Toward Female Patients , 1981, Medical care.

[48]  Data collection in decision‐making: a study in general practice , 1987, Medical education.

[49]  T. Wallsten.  Physician and Medical Student Bias in Evaluating Diagnostic Information , 1981, Medical decision making : an international journal of the Society for Medical Decision Making.

[50]  S. Damrosch.  Nursing students' assessments of behaviorally self-blaming rape victims. , 1985, Nursing research.

[51]  A. Elstein,et al.  Rationale for Physicians' Decisions to Refer Obese Patients , 1985, Medical decision making : an international journal of the Society for Medical Decision Making.

[52]  L. Shulman,et al.  Medical Problem Solving: An Analysis of Clinical Reasoning , 1978 .

[53]  S. Gabbe,et al.  Decision Analysis of High-Risk Patient Referral , 1984, Obstetrics and gynecology.

[54]  R. Brook,et al.  Derivation of clinical indications for carotid endarterectomy by an expert panel. , 1987, American journal of public health.

[55]  B. Starfield,et al.  Surgical decision making and operative rates. , 1984, Archives of surgery.

[56]  J A Farrow.  The use of vignette analysis of dangerous driving situations involving alcohol to differentiate adolescent DWI offenders and high school drivers. , 1987, The American journal of drug and alcohol abuse.

[57]  L D Gruppen,et al.  The influence of general and case-related experience on primary care treatment decision making. , 1988, Archives of internal medicine.

[58]  J P Kassirer,et al.  Dealing with uncertainty, risks, and tradeoffs in clinical decisions. A cognitive science approach. , 1988, Annals of internal medicine.

[59]  S. Damrosch.  Graduate nursing students' attitudes toward sexually active older persons. , 1984, The Gerontologist.

[60]  G. Norman,et al.  A comparison of behaviour on simulated patients and patient management problems , 1981, Medical education.

[61]  J G Dolan,et al.  An Eualuation of Clinicians' Subjective Prior Probability Estimates , 1986, Medical decision making : an international journal of the Society for Medical Decision Making.

[62]  A. Tversky,et al.  On the elicitation of preferences for alternative therapies. , 1982, The New England journal of medicine.

[63]  H. Becker,et al.  The Use of Vignettes in Survey Research , 1978 .

[64]  W. Holzemer,et al.  A Validation Study: Simulations as a Measure of Nurse Practitioners' Problem‐Solving Skills , 1981, Nursing research.

[65]  C. Harris,et al.  Case History Questionnaires in the Study of Doctors' Use of Resources: Are They Measuring What We Want? , 1985, Medical care.

[66]  M. Becker,et al.  Physician behavior as a determinant of utilization patterns: the case of abortion. , 1978, American journal of public health.

[67]  A comparison of initial diagnostic hypotheses of medical students and internists. , 1984, Journal of medical education.

[68]  B. Leake,et al.  Differences in the numbers and costs of tests ordered by internists, family physicians, and psychiatrists. , 1984, Inquiry : a journal of medical care organization, provision and financing.

[69]  S. Mazzuca,et al.  The role of physicians' knowledge and attitudes in effective diabetes management. , 1984, Social science & medicine.

[70]  L. Doyal,et al.  INTERPRETING SURVIVAL RATES FOR THE TREATMENT OF DECOMPENSATED DIABETES: ARE WE SAVING TOO MANY LIVES? , 1987, The Lancet.

[71]  W. Tierney,et al.  Jehovah's Witnesses and Blood Transfusion: Physicians' Attitudes and Legal Precedents , 1984, Southern medical journal.

[72]  E. Graaff,et al.  Validation of a new measure of clinical problem‐solving , 1987, Medical education.

[73]  A S Elstein,et al.  Comparison of physicians' decisions regarding estrogen replacement therapy for menopausal women and decisions derived from a decision analytic model. , 1986, The American journal of medicine.

[74]  R. E. Helfer,et al.  Measuring the process of solving clinical diagnostic problems , 1971, British journal of medical education.

[75]  R. Haynes,et al.  Computer-aided quality assurance. A critical appraisal. , 1987, Archives of internal medicine.

[76]  J. Gillis,et al.  A Cross-National Study of Drug Treatment Decisions in Psychiatry , 1982, Medical decision making : an international journal of the Society for Medical Decision Making.

[77]  R S Wigton,et al.  How Physicians Use Clinical Information in Diagnosing Pulmonary Embolism , 1986, Medical decision making : an international journal of the Society for Medical Decision Making.