The importance of interpretability and visualization in machine learning for applications in medicine and health care

In a short period of time, many areas of science have made a sharp transition towards data-dependent methods. In some cases, this process has been enabled by simultaneous advances in data acquisition and the development of networked system technologies. This new situation is particularly clear in the life sciences, where data overabundance has sparked a flurry of new methodologies for data management and analysis. This can be seen as a perfect scenario for the use of machine learning and computational intelligence techniques to address problems in which more traditional data analysis approaches might struggle. But, this scenario also poses some serious challenges. One of them is model interpretability and explainability, especially for complex nonlinear models. In some areas such as medicine and health care, not addressing such challenge might seriously limit the chances of adoption, in real practice, of computer-based systems that rely on machine learning and computational intelligence methods for data analysis. In this paper, we reflect on recent investigations about the interpretability and explainability of machine learning methods and discuss their impact on medicine and health care. We pay specific attention to one of the ways in which interpretability and explainability in this context can be addressed, which is through data and model visualization. We argue that, beyond improving model interpretability as a goal in itself, we need to integrate the medical experts in the design of data analysis interpretation strategies. Otherwise, machine learning is unlikely to become a part of routine clinical and health care practice.

[1]  I. Guyon,et al.  The Higgs Machine Learning Challenge , 2015 .

[2]  M. Schatz,et al.  Big Data: Astronomical or Genomical? , 2015, PLoS biology.

[3]  Tim Miller,et al.  Explainable AI: Beware of Inmates Running the Asylum Or: How I Learnt to Stop Worrying and Love the Social and Behavioural Sciences , 2017, ArXiv.

[4]  Alfredo Vellido,et al.  Machine learning in critical care: state-of-the-art and a sepsis case study , 2018, BioMedical Engineering OnLine.

[5]  Fei Wang,et al.  Deep learning for healthcare: review, opportunities and challenges , 2018, Briefings Bioinform..

[6]  Hetan Shah The DeepMind debacle demands dialogue on data , 2017, Nature.

[7]  Jimeng Sun,et al.  RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Records , 2018, IEEE Transactions on Visualization and Computer Graphics.

[8]  J V Tu,et al.  Advantages and disadvantages of using artificial neural networks versus logistic regression for predicting medical outcomes. , 1996, Journal of clinical epidemiology.

[9]  Sabina Leonelli,et al.  Data-Centric Biology: A Philosophical Study , 2016 .

[10]  David Weinberger,et al.  Accountability of AI Under the Law: The Role of Explanation , 2017, ArXiv.

[11]  Saad Zafar,et al.  Machine learning based decision support systems (DSS) for heart disease diagnosis: a review , 2017, Artificial Intelligence Review.

[12]  S. Brunak,et al.  Mining electronic health records: towards better research applications and clinical care , 2012, Nature Reviews Genetics.

[13]  Margarida Julià-Sapé,et al.  Strategies for annotation and curation of translational databases: the eTUMOUR project , 2012, Database J. Biol. Databases Curation.

[14]  E. Berner,et al.  Overconfidence as a cause of diagnostic error in medicine. , 2008, The American journal of medicine.

[15]  Daniel A. Keim,et al.  Visual Analytics: Scope and Challenges , 2008, Visual Data Mining.

[16]  Michael Riegler,et al.  Mimir: an automatic reporting and reasoning system for deep learning based analysis in the medical domain , 2018, MMSys.

[17]  Jimeng Sun,et al.  RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism , 2016, NIPS.

[18]  L. Stein The case for cloud computing in genome informatics , 2010, Genome Biology.

[19]  Marc Berg,et al.  Viewpoint Paper: Some Unintended Consequences of Information Technology in Health Care: The Nature of Patient Care Information System-related Errors , 2003, J. Am. Medical Informatics Assoc..

[20]  Daniel A. Keim,et al.  What you see is what you can change: Human-centered machine learning by interactive visualization , 2017, Neurocomputing.

[21]  Or Biran,et al.  Explanation and Justification in Machine Learning : A Survey Or , 2017 .

[22]  Joshua A. Kroll The fallacy of inscrutability , 2018, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[23]  Svetha Venkatesh,et al.  $\mathtt {Deepr}$: A Convolutional Net for Medical Records , 2016, IEEE Journal of Biomedical and Health Informatics.

[24]  Xiting Wang,et al.  Towards better analysis of machine learning models: A visual analytics perspective , 2017, Vis. Informatics.

[25]  Bolei Zhou,et al.  Expert identification of visual primitives used by CNNs during mammogram classification , 2018, Medical Imaging.

[26]  Amina Adadi,et al.  Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI) , 2018, IEEE Access.

[27]  Li Li,et al.  Deep Patient: An Unsupervised Representation to Predict the Future of Patients from the Electronic Health Records , 2016, Scientific Reports.

[28]  Johan A. K. Suykens,et al.  Explaining Support Vector Machines: A Color Based Nomogram , 2016, PloS one.

[29]  Yan Liu,et al.  Interpretable Deep Models for ICU Outcome Prediction , 2016, AMIA.

[30]  Guang-Zhong Yang,et al.  Deep Learning for Health Informatics , 2017, IEEE Journal of Biomedical and Health Informatics.

[31]  Naftali Tishby,et al.  Opening the Black Box of Deep Neural Networks via Information , 2017, ArXiv.

[32]  Parisa Rashidi,et al.  Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis , 2017, IEEE Journal of Biomedical and Health Informatics.

[33]  Andrew Zisserman,et al.  Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps , 2013, ICLR.

[34]  Stephan Dreiseitl,et al.  Do physicians value decision support? A look at the effect of decision support systems on physician opinion , 2005, Artif. Intell. Medicine.

[35]  Margarida Julià-Sapé,et al.  Automated Quality Control for Proton Magnetic Resonance Spectroscopy Data Using Convex Non-negative Matrix Factorization , 2016, IWBBIO.

[36]  Ronald Jackups Deep Learning Makes Its Way to the Clinical Laboratory. , 2017, Clinical chemistry.

[37]  Anne E Carpenter,et al.  Opportunities and obstacles for deep learning in biology and medicine , 2017, bioRxiv.

[38]  C Arús,et al.  Robust discrimination of glioblastomas from metastatic brain tumors on the basis of single‐voxel 1H MRS , 2012, NMR in biomedicine.

[39]  Hang Su,et al.  Towards Interpretable Deep Neural Networks by Leveraging Adversarial Examples , 2017, ArXiv.

[40]  Yan Liu,et al.  Distilling Knowledge from Deep Networks with Applications to Healthcare Domain , 2015, ArXiv.

[41]  Zhen Li,et al.  Towards Better Analysis of Deep Convolutional Neural Networks , 2016, IEEE Transactions on Visualization and Computer Graphics.

[42]  Corinne Cath Governing artificial intelligence: ethical, legal and technical opportunities and challenges , 2018, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[43]  C. Pal,et al.  Deep Learning: A Primer for Radiologists. , 2017, Radiographics : a review publication of the Radiological Society of North America, Inc.

[44]  Lluís A. Belanche Muñoz,et al.  Outlier exploration and diagnostic classification of a multi-centre 1H-MRS brain tumour database , 2009, Neurocomputing.

[45]  Andreas Holzinger,et al.  Interactive machine learning for health informatics: when do we need the human-in-the-loop? , 2016, Brain Informatics.

[46]  Artificial Intelligence : Potential Benefits and Ethical Considerations , 2016 .

[47]  Yongxin Zhu,et al.  Exploring High Efficiency Hardware Accelerator for the Key Algorithm of Square Kilometer Array Telescope Data Processing , 2017, 2017 IEEE 25th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM).

[48]  Nilmini Wickramasinghe,et al.  Deepr: A Convolutional Net for Medical Records , 2016, ArXiv.

[49]  Thomas Villmann,et al.  Biomedical data analysis in translational research : Integration of expert knowledge and interpretable models , 2017, ESANN 2017.

[50]  Fenglong Ma,et al.  Dipole: Diagnosis Prediction in Healthcare via Attention-based Bidirectional Recurrent Neural Networks , 2017, KDD.

[51]  Seth Flaxman,et al.  European Union Regulations on Algorithmic Decision-Making and a "Right to Explanation" , 2016, AI Mag..

[52]  Luciano Floridi,et al.  Why a Right to Explanation of Automated Decision-Making Does Not Exist in the General Data Protection Regulation , 2017 .

[53]  T. Hoff Deskilling and adaptation among primary care physicians using two work innovations , 2011, Health care management review.

[54]  O. Stegle,et al.  Deep learning for computational biology , 2016, Molecular systems biology.

[55]  Paulo J. G. Lisboa,et al.  Seeing is believing: The importance of visualization in real-world machine learning applications , 2011, ESANN.

[56]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[57]  V. Marx Biology: The big challenges of big data , 2013, Nature.

[58]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[59]  MICHAEL PRESS OF ROBOTS AND RULES : AUTONOMOUS WEAPON SYSTEMS IN THE LAW OF ARMED CONFLICT , 2018 .

[60]  Jeffrey Dean,et al.  Scalable and accurate deep learning with electronic health records , 2018, npj Digital Medicine.

[61]  May D. Wang,et al.  Interpretable Predictions of Clinical Outcomes with An Attention-based Recurrent Neural Network , 2017, BCB.

[62]  Paulo J. G. Lisboa,et al.  How to find simple and accurate rules for viral protease cleavage specificities , 2009, BMC Bioinformatics.

[63]  Neil Davison,et al.  A legal perspective: Autonomous weapon systems under international humanitarian law , 2018 .

[64]  Mike Wu,et al.  Beyond Sparsity: Tree Regularization of Deep Models for Interpretability , 2017, AAAI.

[65]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[66]  F. Cabitza,et al.  Unintended Consequences of Machine Learning in Medicine , 2017, JAMA.

[67]  Paulo J. G. Lisboa,et al.  Making machine learning models interpretable , 2012, ESANN.

[68]  Thomas Blaschke,et al.  The rise of deep learning in drug discovery. , 2018, Drug discovery today.

[69]  M. Julià-Sapé,et al.  A Multi-Centre, Web-Accessible and Quality Control-Checked Database of in vivo MR Spectra of Brain Tumour Patients , 2006, Magnetic Resonance Materials in Physics, Biology and Medicine.

[70]  D. Britton,et al.  How to deal with petabytes of data: the LHC Grid project , 2014, Reports on progress in physics. Physical Society.

[71]  Kayvan Najarian,et al.  Transforming big data into computational models for personalized medicine and health care , 2016, Dialogues in clinical neuroscience.

[72]  Lin Yang,et al.  MDNet: A Semantically and Visually Interpretable Medical Image Diagnosis Network , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[73]  Swarup Roy,et al.  Big Data Analytics in Bioinformatics: A Machine Learning Perspective , 2015, ArXiv.

[74]  Paulo J. G. Lisboa,et al.  Bioinformatics and medicine in the era of deep learning , 2018, ESANN.

[75]  Been Kim,et al.  Towards A Rigorous Science of Interpretable Machine Learning , 2017, 1702.08608.

[76]  Nuno Pombo,et al.  Knowledge discovery in clinical decision support systems for pain management: A systematic review , 2014, Artif. Intell. Medicine.

[77]  C. Langlotz,et al.  Deep Learning in Neuroradiology , 2018, American Journal of Neuroradiology.

[78]  Alex Zhavoronkov,et al.  Applications of Deep Learning in Biomedicine. , 2016, Molecular pharmaceutics.