Unintended Consequences of Machine Learning in Medicine

Over the past decade, machine learning techniques have made substantial advances in many domains. In health care, global interest in the potential of machine learning has increased; for example, a deep learning algorithm has shown high accuracy in detecting diabetic retinopathy.1 There have been suggestions that machine learning will drive changes in health care within a few years, specifically in medical disciplines that require more accurate prognostic models (eg, oncology) and those based on pattern recognition (eg, radiology and pathology). However, comparative studies on the effectiveness of machine learning–based decision support systems (ML-DSS) in medicine are lacking, especially regarding the effects on health outcomes. Moreover, the introduction of new technologies in health care has not always been straightforward or without unintended and adverse effects.2 In this Viewpoint we consider the potential unintended consequences that may result from the application of ML-DSS in clinical practice.

[1]  Julian D. Olden,et al.  Illuminating the “black box”: a randomization approach for understanding variable contributions in artificial neural networks , 2002 .

[2]  Douglas B. Fridsma,et al.  Research Paper: Computer Decision Support as a Source of Interpretation Error: The Case of Electrocardiograms , 2003, J. Am. Medical Informatics Assoc..

[3]  Marc Berg,et al.  Viewpoint Paper: Some Unintended Consequences of Information Technology in Health Care: The Nature of Patient Care Information System-related Errors , 2003, J. Am. Medical Informatics Assoc..

[4]  Oguzhan Alagoz,et al.  Informatics in radiology: comparison of logistic regression and artificial neural network models in breast cancer risk estimation. , 2010, Radiographics : a review publication of the Radiological Society of North America, Inc.

[5]  W. Moon,et al.  Comparison of two software versions of a commercially available computer-aided detection (CAD) system for detecting breast cancer , 2010, Acta radiologica.

[6]  T. Hoff Deskilling and adaptation among primary care physicians using two work innovations , 2011, Health care management review.

[7]  Lorenzo Strigini,et al.  How to Discriminate between Computer-Aided and Computer-Hindered Decisions , 2013, Medical decision making : an international journal of the Society for Medical Decision Making.

[8]  Marc Thilo Figge,et al.  Automated Classification of Circulating Tumor Cells and the Impact of Interobsever Variability on Classifier Training and Performance , 2015, Journal of immunology research.

[9]  Chengqi Zhang,et al.  Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , 2015, KDD.

[10]  Johannes Gehrke,et al.  Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission , 2015, KDD.

[11]  Subhashini Venugopalan,et al.  Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. , 2016, JAMA.

[12]  M. Tinetti,et al.  Treatment for Multiple Acute Cardiopulmonary Conditions in Older Adults Hospitalized with Pneumonia, Chronic Obstructive Pulmonary Disease, or Heart Failure , 2016, Journal of the American Geriatrics Society.

[13]  L. McDonald,et al.  Unintended consequences of machine learning in medicine? , 2017, F1000Research.