Bayesian LSTMs in medicine

The medical field stands to see significant benefits from the recent advances in deep learning. Knowing the uncertainty in the decision made by any machine learning algorithm is of utmost importance for medical practitioners. This study demonstrates the utility of using Bayesian LSTMs for classification of medical time series. Four medical time series datasets are used to show the accuracy improvement Bayesian LSTMs provide over standard LSTMs. Moreover, we show cherry-picked examples of confident and uncertain classifications of the medical time series. With simple modifications of the common practice for deep learning, significant improvements can be made for the medical practitioner and patient.

[1]  Aram Galstyan,et al.  Multitask learning and benchmarking with clinical time series data , 2017, Scientific Data.

[2]  Zoubin Ghahramani,et al.  A Theoretically Grounded Application of Dropout in Recurrent Neural Networks , 2015, NIPS.

[3]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[4]  Jeffrey M. Hausdorff,et al.  Physionet: Components of a New Research Resource for Complex Physiologic Signals". Circu-lation Vol , 2000 .

[5]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[6]  Zoubin Ghahramani,et al.  Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference , 2015, ArXiv.

[7]  G.B. Moody,et al.  The impact of the MIT-BIH Arrhythmia Database , 2001, IEEE Engineering in Medicine and Biology Magazine.

[8]  J. Pickard,et al.  Continuous assessment of the cerebral vasomotor reactivity in head injury. , 1997, Neurosurgery.

[9]  David J. C. MacKay,et al.  A Practical Bayesian Framework for Backpropagation Networks , 1992, Neural Computation.

[10]  Charles Elkan,et al.  Learning to Diagnose with LSTM Recurrent Neural Networks , 2015, ICLR.

[11]  B. Jennett,et al.  ASSESSMENT OF OUTCOME AFTER SEVERE BRAIN DAMAGE A Practical Scale , 1975, The Lancet.

[12]  Qiao Li,et al.  Classification of normal/abnormal heart sound recordings: The PhysioNet/Computing in Cardiology Challenge 2016 , 2016, 2016 Computing in Cardiology Conference (CinC).

[13]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[14]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[15]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[16]  Yoshua Bengio,et al.  Architectural Complexity Measures of Recurrent Neural Networks , 2016, NIPS.

[17]  J. Ramon,et al.  Machine learning techniques to examine large patient databases. , 2009, Best practice & research. Clinical anaesthesiology.

[18]  B. Cabella,et al.  Complexity of brain signals is associated with outcome in preterm infants , 2017, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[19]  Yann LeCun,et al.  The mnist database of handwritten digits , 2005 .

[20]  Yarin Gal,et al.  Uncertainty in Deep Learning , 2016 .

[21]  Peter Szolovits,et al.  A Multivariate Timeseries Modeling Approach to Severity of Illness Assessment and Forecasting in ICU with Sparse, Heterogeneous Clinical Data , 2015, AAAI.

[22]  Filip De Turck,et al.  Time series classification for the prediction of dialysis in critically ill patients using echo statenetworks , 2013, Eng. Appl. Artif. Intell..

[23]  Igor Kononenko,et al.  Machine learning for medical diagnosis: history, state of the art and perspective , 2001, Artif. Intell. Medicine.

[24]  Jan Ramon,et al.  Gaussian processes for prediction in intensive care , 2006 .

[25]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[26]  Shih-Chii Liu,et al.  Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequences , 2016, NIPS.

[27]  Constantin F. Aliferis,et al.  Medical decision support using machine learning for early detection of late-onset neonatal sepsis , 2014, J. Am. Medical Informatics Assoc..

[28]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[29]  Hong Yu,et al.  Bidirectional RNN for Medical Event Detection in Electronic Health Records , 2016, NAACL.

[30]  Gari D Clifford,et al.  Automated signal quality assessment of mobile phone-recorded heart sound signals , 2016, Journal of medical engineering & technology.

[31]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[32]  Yann LeCun,et al.  Transforming Neural-Net Output Levels to Probability Distributions , 1990, NIPS.

[33]  Jen-Tzung Chien,et al.  Bayesian recurrent neural network language model , 2014, 2014 IEEE Spoken Language Technology Workshop (SLT).

[34]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[35]  Glen Wright Colopy,et al.  Health Informatics via Machine Learning for the Clinical Management of Patients , 2015, Yearbook of Medical Informatics.

[36]  Walter F. Stewart,et al.  Doctor AI: Predicting Clinical Events via Recurrent Neural Networks , 2015, MLHC.

[37]  Roberto Cipolla,et al.  Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding , 2015, BMVC.

[38]  Aaron C. Courville,et al.  Recurrent Batch Normalization , 2016, ICLR.

[39]  Willis J. Tompkins,et al.  A Real-Time QRS Detection Algorithm , 1985, IEEE Transactions on Biomedical Engineering.

[40]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[41]  G. Lightbody,et al.  EEG-based neonatal seizure detection with Support Vector Machines , 2011, Clinical Neurophysiology.

[42]  Alex Kendall,et al.  What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? , 2017, NIPS.

[43]  Qiao Li,et al.  An open access database for the evaluation of heart sound algorithms , 2016, Physiological measurement.

[44]  Rico Sennrich,et al.  Edinburgh Neural Machine Translation Systems for WMT 16 , 2016, WMT.

[45]  Zhongheng Zhang,et al.  When doctors meet with AlphaGo: potential application of machine learning to clinical medicine. , 2016, Annals of translational medicine.

[46]  Kris K. Hauser,et al.  Artificial intelligence framework for simulating clinical decision-making: A Markov decision process approach , 2013, Artif. Intell. Medicine.

[47]  Alex Graves,et al.  Practical Variational Inference for Neural Networks , 2011, NIPS.