Do physicians value decision support? A look at the effect of decision support systems on physician opinion

OBJECTIVE Clinical decision support systems are on the verge of becoming routine software tools in clinical settings. We investigate the question of how physicians react when faced with decision support suggestions that contradict their own diagnoses. METHODOLOGY We used a study design involving 52 volunteer dermatologists who each rated the malignancy of 25 lesion images on an ordinal scale and gave a dichotomous excise/no excise recommendation for each lesion image. After seeing the system's rating and excise suggestions, the physicians could revise their initial recommendations. RESULTS We observed that in 24% of the cases in which the physicians' diagnoses did not match those of the decision support system, the physicians changed their diagnoses. There was a slight but significant negative correlation between susceptibility to change and experience level of the physicians. Physicians were significantly less likely to follow the decision system's recommendations when they were confident of their initial diagnoses. No differences between the physicians' inclinations to following excise versus no excise recommendations could be observed. CONCLUSION These results indicate that physicians are quite susceptible to accepting the recommendations of decision support systems, and that quality assurance and validation of such systems is therefore of paramount importance.

[1]  D G Altman,et al.  What do we mean by validating a prognostic model? , 2000, Statistics in medicine.

[2]  B S Todd,et al.  Case-Based Explanation for Medical Diagnostic Programs, with an Example from Gynaecology , 1994, Methods of Information in Medicine.

[3]  Chris D. Nugent,et al.  Evaluation of inherent performance of intelligent medical decision support systems: utilising neural networks as an example , 2003, Artif. Intell. Medicine.

[4]  R G Neville,et al.  Lessons from a randomized controlled trial designed to evaluate computer decision support software to improve the management of asthma , 2001, Medical informatics and the Internet in medicine.

[5]  Charles P. Friedman,et al.  Evaluation Methods in Medical Informatics , 1997, Computers and Medicine.

[6]  Edward H. Shortliffe,et al.  Computer-based medical consultations, MYCIN , 1976 .

[7]  Casimir A. Kulikowski,et al.  A Model-Based Method for Computer-Aided Medical Decision-Making , 1978, Artif. Intell..

[8]  R. Haynes,et al.  Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. , 1998, JAMA.

[9]  Bonnie Kaplan,et al.  Evaluating informatics applications - clinical decision support systems literature review , 2001, Int. J. Medical Informatics.

[10]  E. Berner,et al.  Clinical Decision Support Systems: Theory and Practice , 1998 .

[11]  L. Mango Computer-assisted cervical cancer screening using neural networks. , 1994, Cancer Letters.

[12]  F E Masarie,et al.  Quick medical reference (QMR) for diagnostic assistance. , 1986, M.D.Computing.

[13]  H. E. Pople,et al.  Internist-I, an Experimental Computer-Based Diagnostic Consultant for General Internal Medicine , 1982 .

[14]  J Wyatt,et al.  Quantitative evaluation of clinical software, exemplified by decision support systems. , 1997, International journal of medical informatics.

[15]  H. Kittler,et al.  Risks and benefits of sequential imaging of melanocytic skin lesions in patients with multiple atypical nevi. , 2001, Archives of dermatology.

[16]  David W. Bates,et al.  Synthesis of Research Paper: Ten Commandments for Effective Clinical Decision Support: Making the Practice of Evidence-based Medicine a Reality , 2003, J. Am. Medical Informatics Assoc..

[17]  D. Spiegelhalter,et al.  Evaluating medical expert systems: what to test and how? , 1990, Medical informatics = Medecine et informatique.

[18]  Peter J. Haug,et al.  Decision support in medicine: lessons from the HELP system , 2003, Int. J. Medical Informatics.

[19]  Randolph A. Miller,et al.  Review: Medical Diagnostic Decision Support Systems - Past, Present, And Future: A Threaded Bibliography and Brief Commentary , 1994, J. Am. Medical Informatics Assoc..

[20]  S. Weiss,et al.  Glaucoma consultation by computer. , 1978, Computers in biology and medicine.

[21]  R. Haynes,et al.  Effects of Computer-based Clinical Decision Support Systems on Clinician Performance and Patient Outcome: A Critical Appraisal of Research , 1994, Annals of Internal Medicine.

[22]  Eta S. Berner,et al.  Research Paper: Influence of Case and Physician Characteristics on Perceptions of Decision Support Systems , 1999, J. Am. Medical Informatics Assoc..

[23]  Timothy M. Franz,et al.  Enhancement of clinicians' diagnostic reasoning by computer-based consultation: a multisite study of 2 systems. , 1999, JAMA.

[24]  H. Kittler,et al.  Diagnostic accuracy of dermoscopy. , 2002, The Lancet. Oncology.

[25]  G. Barnett,et al.  DXplain. An evolving diagnostic decision-support system. , 1987, JAMA.

[26]  M H Trivedi,et al.  Development and Implementation of Computerized Clinical Guidelines: Barriers and Solutions , 2002, Methods of Information in Medicine.

[27]  E.B Reategui,et al.  Combining a neural network with case-based reasoning in a diagnostic system , 1997, Artif. Intell. Medicine.

[28]  Fredric M. Wolf,et al.  Are Clinicians Correct When They Believe They are Correct? Implications for Medical Decision Support , 2001, MedInfo.

[29]  Bonnie Kaplan,et al.  Clinical decision support systems for the practice of evidence-based medicine. , 2001, Journal of the American Medical Informatics Association : JAMIA.

[30]  Jacobus Ridderikhoff,et al.  Who is afraid of the system? Doctors' attitude towards diagnostic systems , 1999, Int. J. Medical Informatics.

[31]  D. Heckerman,et al.  Toward Normative Expert Systems: Part I The Pathfinder Project , 1992, Methods of Information in Medicine.

[32]  Fredric M. Wolf,et al.  Effects of a decision support system on the diagnostic accuracy of users: a preliminary report. , 1996, Journal of the American Medical Informatics Association : JAMIA.

[33]  Lucila Ohno-Machado,et al.  A Comparison of Machine Learning Methods for the Diagnosis of Pigmented Skin Lesions , 2001, J. Biomed. Informatics.

[34]  Klaus-Peter Adlassnig,et al.  Development and retrospective evaluation of Hepaxpert-I: a routinely-used expert system for interpretive analysis of hepatitis A and B serologic findings , 1995, Artif. Intell. Medicine.

[35]  William M. Tierney,et al.  Improving clinical decisions and outcomes with information: a review , 2001, Int. J. Medical Informatics.

[36]  Christian Nøhr,et al.  The evaluation of expert diagnostic systems. How to assess outcomes and quality parameters? , 1994, Artif. Intell. Medicine.

[37]  Phei Lang ChangHANG,et al.  Evaluation of a Decision-support System for Preoperative Staging of Prostate Cancer , 1999, Medical decision making : an international journal of the Society for Medical Decision Making.

[38]  K Wolff,et al.  Computer-aided epiluminescence microscopy of pigmented skin lesions: the value of clinical data for the classification process , 2000, Melanoma research.

[39]  Eta S. Berner,et al.  Effects of a decision support system on physicians' diagnostic performance. , 1999, Journal of the American Medical Informatics Association : JAMIA.