A Deep Representation of Longitudinal EMR Data Used for Predicting Readmission to the ICU and Describing Patients-at-Risk

To compare different deep learning architectures for predicting the risk of readmission within 30 days of discharge from the intensive care unit (ICU). The interpretability of attention-based models is leveraged to describe patients-at-risk. Several deep learning architectures making use of attention mechanisms, recurrent layers, neural ordinary differential equations (ODEs), and medical concept embeddings with time-aware attention were trained using publicly available electronic medical record data (MIMIC-III) associated with 45,298 ICU stays for 33,150 patients. Bayesian inference was used to compute the posterior over weights of an attention-based model. Odds ratios associated with an increased risk of readmission were computed for static variables. Diagnoses, procedures, medications, and vital signs were ranked according to the associated risk of readmission. A recurrent neural network, with time dynamics of code embeddings computed by neural ODEs, achieved the highest average precision of 0.331 (AUROC: 0.739, F 1 -Score: 0.372). Predictive accuracy was comparable across neural network architectures. Groups of patients at risk included those suffering from infectious complications, with chronic or progressive conditions, and for whom standard medical care was not suitable. Attention-based networks may be preferable to recurrent networks if an interpretable model is required, at only marginal cost in predictive accuracy.

[1]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[2]  A. Rosenberg,et al.  Patients readmitted to ICUs* : a systematic review of risk factors and outcomes. , 2000, Chest.

[3]  J. Zimmerman,et al.  The Association Between ICU Readmission Rate and Patient Outcomes* , 2013, Critical care medicine.

[4]  S. T. Buckland,et al.  An Introduction to the Bootstrap. , 1994 .

[5]  Gary M. Weiss,et al.  Cost-Sensitive Learning vs. Sampling: Which is Best for Handling Unbalanced Classes with Unequal Error Costs? , 2007, DMIN.

[6]  Svetha Venkatesh,et al.  Resset: A Recurrent Model for Sequence of Sets with Applications to Electronic Medical Records , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[7]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[8]  Lina De Geer,et al.  Risk prediction of ICU readmission in a mixed surgical and medical population , 2015, Journal of Intensive Care.

[9]  John Platt,et al.  Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods , 1999 .

[10]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[11]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[12]  Susan Hutfless,et al.  Mining high-dimensional administrative claims data to predict early hospital readmissions , 2014, J. Am. Medical Informatics Assoc..

[13]  M. Silverstein,et al.  Risk Factors for 30-Day Hospital Readmission in Patients ≥65 Years of Age , 2008, Proceedings.

[14]  W J Sibbald,et al.  Patients readmitted to the intensive care unit during the same hospitalization: clinical features and outcomes. , 1998, Critical care medicine.

[15]  Jimeng Sun,et al.  RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism , 2016, NIPS.

[16]  Michael V. McConnell,et al.  Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning , 2017, Nature Biomedical Engineering.

[17]  Peter Szolovits,et al.  MIMIC-III, a freely accessible critical care database , 2016, Scientific Data.

[18]  Li Li,et al.  Deep Patient: An Unsupervised Representation to Predict the Future of Patients from the Electronic Health Records , 2016, Scientific Reports.

[19]  Robert Tibshirani,et al.  An Introduction to the Bootstrap , 1994 .

[20]  Yee Whye Teh,et al.  Augmented Neural ODEs , 2019, NeurIPS.

[21]  Michael C. Mozer,et al.  Discrete Event, Continuous Time RNNs , 2017, ArXiv.

[22]  A. Majeed,et al.  Identifying Patients at High Risk of Emergency Hospital Admissions: A Logistic Regression Analysis , 2006 .

[23]  Yan Liu,et al.  Recurrent Neural Networks for Multivariate Time Series with Missing Values , 2016, Scientific Reports.

[24]  R. Campbell,et al.  Analysis and prediction of unplanned intensive care unit readmission using recurrent neural networks with long short-term memory , 2018, bioRxiv.

[25]  Tosha B. Wetterneck,et al.  Hospital Readmission in General Medicine Patients: A Prediction Model , 2009, Journal of General Internal Medicine.

[26]  Jeffrey Dean,et al.  Scalable and accurate deep learning with electronic health records , 2018, npj Digital Medicine.

[27]  Michelle D. Miller,et al.  Malnutrition and its association with readmission and death within 7 days and 8–180 days postdischarge in older patients: a prospective observational study , 2017, BMJ Open.

[28]  Geoffrey E. Hinton,et al.  Keeping the neural networks simple by minimizing the description length of the weights , 1993, COLT '93.

[29]  Beng Chin Ooi,et al.  Medical Concept Embedding with Time-Aware Attention , 2018, IJCAI.

[30]  R. Oye,et al.  Patterns of resource consumption in medical intensive care. , 1991, Chest.

[31]  A. Serpa Neto,et al.  Readmission to the Intensive Care Unit: Incidence, Risk Factors, Resource Use, and Outcomes. A Retrospective Cohort Study , 2017, Annals of the American Thoracic Society.

[32]  W. Ghali,et al.  A prospective observational study of physician handoff for intensive-care-unit-to-ward patient transfers. , 2011, The American journal of medicine.

[33]  R. Hayward,et al.  Who bounces back? Physiologic and other predictors of intensive care unit readmission , 2001, Critical care medicine.

[34]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.

[35]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[36]  J. L. Gall,et al.  APACHE II--a severity of disease classification system. , 1986, Critical care medicine.

[37]  S. Lemeshow,et al.  A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. , 1993, JAMA.

[38]  A. Garland,et al.  Epidemiology of critically ill patients in intensive care units: a population-based observational study , 2013, Critical Care.

[39]  Patricia M Davidson,et al.  Severity of illness and risk of readmission to intensive care: a meta-analysis. , 2009, Resuscitation.

[40]  R. Bellomo,et al.  Readmissions to Intensive Care: A Prospective Multicenter Study in Australia and New Zealand* , 2017, Critical care medicine.

[41]  Rob J. Bosman,et al.  Readmission of ICU patients: A quality indicator?☆ , 2017, Journal of critical care.

[42]  W. Youden,et al.  Index for rating diagnostic tests , 1950, Cancer.

[43]  Jonathan H. Chen,et al.  Machine Learning and Prediction in Medicine - Beyond the Peak of Inflated Expectations. , 2017, The New England journal of medicine.

[44]  Andreas Spanias,et al.  Attend and Diagnose: Clinical Time Series Analysis using Attention Models , 2017, AAAI.

[45]  James H. Harrison,et al.  Patient2Vec: A Personalized Interpretable Deep Representation of the Longitudinal Electronic Health Record , 2018, IEEE Access.

[46]  Hae Mi Choe,et al.  The relationship between diabetes mellitus and 30-day readmission rates , 2017, Clinical Diabetes and Endocrinology.

[47]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[48]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[49]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[50]  Gari D. Clifford,et al.  A New Severity of Illness Scale Using a Subset of Acute Physiology and Chronic Health Evaluation Data Elements Shows Comparable Predictive Accuracy* , 2013, Critical care medicine.

[51]  Takaya Saito,et al.  The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets , 2015, PloS one.

[52]  Tim Oates,et al.  Time series classification from scratch with deep neural networks: A strong baseline , 2016, 2017 International Joint Conference on Neural Networks (IJCNN).

[53]  Zachariah Zhang,et al.  Deep EHR: Chronic Disease Prediction Using Medical Notes , 2018, MLHC.

[54]  Julien Cornebise,et al.  Weight Uncertainty in Neural Networks , 2015, ArXiv.

[55]  Evan G. Wong,et al.  Association of severity of illness and intensive care unit readmission: A systematic review. , 2016, Heart & lung : the journal of critical care.

[56]  Mark Goadrich,et al.  The relationship between Precision-Recall and ROC curves , 2006, ICML.

[57]  J. Garland THE NEW ENGLAND JOURNAL OF MEDICINE , 1977, The Lancet.

[58]  J. Barker A Prospective Observational Study of Physician Handoff for Intensive-Care-Unit-to-Ward Patient Transfers , 2012 .

[59]  David Duvenaud,et al.  Latent ODEs for Irregularly-Sampled Time Series , 2019, ArXiv.

[60]  R. Rosa,et al.  Comparison of Unplanned Intensive Care Unit Readmission Scores: A Prospective Cohort Study , 2015, PloS one.