Debias the environment instead of the judge: an alternative approach to reducing error in diagnostic (and other) judgment

Questions about how to improve human judgment and reasoning are of theoretical and practical interest, notwithstanding the continuing controversy over whether people are "rational". Improving judgment may involve modifying people's processes to fit their environments better, or vice versa. We illustrate the latter approach in a study of diagnostic reasoning in which subjects learned to distinguish two fictitious diseases. Prior findings suggest that people may judge the likelihood of a diagnostic category on the presence or absence of features that are typical of, rather than diagnostic of, the category. We varied the structure of the information provided to subjects without attempting to modify their judgmental processes. In an "independent" format, subjects learned about each disease separately; in a "contrastive" format, information about the two diseases was juxtaposed to highlight distinctive features. Subjects in the two conditions formed different disease concepts. Diagnoses following contrastive training were much closer to the statistically prescribed judgments based on likelihood ratios. Interventions that modify the environment may provide an alternative approach where it is difficult to modify people's processes. Effective design of such interventions is one motivation for directing research toward understanding how task characteristics affect the use of and the outcomes of judgment and reasoning processes.

[1]  J. Klayman Cue discovery in probabilistic environments: Uncertainty and experimentation. , 1988 .

[2]  M. Weinstein,et al.  Clinical Decision Analysis , 1980 .

[3]  H S Barrows,et al.  The diagnostic (problem solving) skill of the neurologist. Experimental studies and their implications for neurological training. , 1972, Archives of neurology.

[4]  V L Patel,et al.  Effects of conventional and problem‐based medical curricula on problem solving , 1991, Academic medicine : journal of the Association of American Medical Colleges.

[5]  Edward E. Smith,et al.  Concepts and concept formation. , 1984, Annual review of psychology.

[6]  L. Shulman,et al.  Medical Problem Solving: An Analysis of Clinical Reasoning , 1978 .

[7]  John R. Anderson The Adaptive Character of Thought , 1990 .

[8]  R. Tamblyn,et al.  Problem-based learning : an approach to medical education , 1980 .

[9]  B. Fischhoff,et al.  Hypothesis Evaluation from a Bayesian Perspective. , 1983 .

[10]  L. Komatsu Recent views of conceptual structure , 1992 .

[11]  J H Galla,et al.  Properdin deficiency with IgA nephropathy. , 1981, The New England journal of medicine.

[12]  P. Wason On the Failure to Eliminate Hypotheses in a Conceptual Task , 1960 .

[13]  L D Gruppen,et al.  Information Gathering and Integration as Sources of Error in Diagnostic Decision Making , 1991, Medical decision making : an international journal of the Society for Medical Decision Making.

[14]  J. Klayman,et al.  Hypothesis testing in rule discovery: Strategy, structure, and content. , 1989 .

[15]  C. Mckenzie The Accuracy of Intuitive Judgment Strategies: Covariation Assessment and Bayesian Inference , 1994, Cognitive Psychology.

[16]  B. Fischhoff,et al.  Reasons for confidence. , 1980 .

[17]  D. Funder,et al.  Errors and mistakes: evaluating the accuracy of social judgment. , 1987, Psychological bulletin.

[18]  V L Patel,et al.  Comparing the effects of problem-based and conventional curricula in an international sample. , 1987, Journal of medical education.

[19]  L. Cohen Can human irrationality be experimentally demonstrated? , 1981, Behavioral and Brain Sciences.

[20]  J. Klayman,et al.  Confirmation, Disconfirmation, and Informa-tion in Hypothesis Testing , 1987 .

[21]  Thomas Kida,et al.  Heuristics and biases: Expertise and task realism in auditing. , 1991 .

[22]  L D Gruppen,et al.  Differential diagnosis and the competing-hypotheses heuristic. A practical approach to judgment under uncertainty and Bayesian probability. , 1985, JAMA.

[23]  John R. Anderson Is human cognition adaptive? , 1991, Behavioral and Brain Sciences.

[24]  M. Hewson,et al.  Conceptual frameworks in preclinical and clinical textbooks , 1986, Medical Education.

[25]  W. Edwards,et al.  Decision Analysis and Behavioral Research , 1986 .

[26]  R. Abelson Decision making and decision theory , 1985 .

[27]  Eric J. Johnson,et al.  Information displays and preference reversals , 1988 .

[28]  V L Patel,et al.  Medical problem‐solving: some questionable assumptions , 1985, Medical education.

[29]  David V. Budescu,et al.  Encoding subjective probabilities: A psychological and psychometric review , 1983 .

[30]  D. Krantz,et al.  The use of statistical heuristics in everyday inductive reasoning , 1983 .

[31]  K. R. Hammond Functionalism and illusionism: Can integration be usefully achieved? , 1990 .

[32]  P M Wortman,et al.  Medical diagnosis: an information-processing approach. , 1972, Computers and biomedical research, an international journal.

[33]  Ryan D. Tweney,et al.  Strategies of Rule Discovery in an Inference Task , 1980 .

[34]  G. Oden Concept, Knowledge, and Thought , 1987 .

[35]  J J Christensen-Szalanski,et al.  Physicians' Misunderstanding of Normal Findings , 1983, Medical decision making : an international journal of the Society for Medical Decision Making.

[36]  D. Krantz,et al.  The effects of statistical training on thinking about everyday problems , 1986, Cognitive Psychology.

[37]  Steven Schwartz,et al.  Medical Thinking: The Psychology of Medical Judgment and Decision Making , 1986 .

[38]  Ruth Beyth-Marom,et al.  How probable is probable? A numerical translation of verbal probability expressions , 1982 .

[39]  T. Pearson,et al.  Direct comparison of the efficacy of intuitive and analytical cognition in expert judgment , 1987, IEEE Transactions on Systems, Man, and Cybernetics.

[40]  Louisa M. Slowiaczek,et al.  Information selection and use in hypothesis testing: What is a good question, and what is a good answer? , 1992, Memory & Cognition.

[41]  P. Politser Medical education for a changing future: new concepts for revising texts , 1987, Medical education.

[42]  Moshe Shaked,et al.  An Elementary Approach To Thinking Under Uncertainty , 1985 .

[43]  J P Kassirer,et al.  Teaching clinical medicine by iterative hypothesis testing. Let's preach what we practice. , 1983, The New England journal of medicine.

[44]  James Shanteau,et al.  Reducing the influence of irrelevant information on experienced decision makers , 1984 .

[45]  M E Doherty,et al.  'Pseudodiagnosticity' in an idealized medical problem-solving environment. , 1982, Journal of medical education.

[46]  Arthur S. Elstein,et al.  Medical Problem Solving , 1990 .

[47]  E. Rosch,et al.  Family resemblances: Studies in the internal structure of categories , 1975, Cognitive Psychology.

[48]  L. Jacoby,et al.  Knowledge and clinical problem‐solving , 1985, Medical education.

[49]  G. Bordage,et al.  Semantic structures and diagnostic thinking of experts and novices. , 1991, Academic medicine : journal of the Association of American Medical Colleges.

[50]  G R Norman,et al.  Problem‐solving skills, solving problems and problem‐based learning , 1988, Medical education.

[51]  G. Norman,et al.  A comparison of behaviour on simulated patients and patient management problems , 1981, Medical education.

[52]  Eric J. Johnson,et al.  The Adaptive Decision-Maker : Effort and Accuracy in Choice , 2022 .

[53]  Georges Bordage,et al.  Propositional Versus Structural Semantic Analysis of Medical Diagnostic Thinking , 1992, Cogn. Sci..

[54]  H. Schmidt,et al.  The psychological basis of problem‐based learning: a review of the evidence , 1992, Academic medicine : journal of the Association of American Medical Colleges.

[55]  Edward E. Smith,et al.  Category-Based Induction , 1990 .

[56]  T. K. Srull,et al.  Handbook of Social Cognition , 1993 .

[57]  H. Jungermann The Two Camps on Rationality , 1983 .

[58]  Maya Bar-Hillel,et al.  Back to base rates. , 1990 .

[59]  G Bordage,et al.  Which medical textbook to read? Emphasizing semantic structures , 1990, Academic medicine : journal of the Association of American Medical Colleges.

[60]  G. Gorry,et al.  Decision analysis and clinical judgment. , 1973, The American journal of medicine.

[61]  M. Nakao,et al.  Numbers are better than words. Verbal specifications of frequency have no place in medicine. , 1983, The American journal of medicine.

[62]  R. Hogarth Beyond discrete biases: Functional and dysfunctional aspects of judgmental heuristics. , 1981 .

[63]  M. Albanese,et al.  Problem‐based Learning: A Review of Literature on Its Outcomes and Implementation Issues , 1993, Academic medicine : journal of the Association of American Medical Colleges.

[64]  Michael J. Saks,et al.  Human Information Processing and Adjudication: Trial by Heuristics , 1980 .

[65]  H S Barrows,et al.  A taxonomy of problem‐based learning methods , 1986, Medical education.

[66]  Clifford R. Mynatt,et al.  The influence of feedback and diagnostic data on pseudodiagnosticity , 1981 .

[67]  D. Bergman,et al.  Using structured medical information to improve students' problem-solving performance. , 1986, Journal of Medical Education.

[68]  J. E. Russo,et al.  The Value of Unit Price Information , 1977 .