Neural Temporal Point Processes For Modelling Electronic Health Records

The modelling of Electronic Health Records (EHRs) has the potential to drive more efficient allocation of healthcare resources, enabling early intervention strategies and advancing personalised healthcare. However, EHRs are challenging to model due to their realisation as noisy, multi-modal data occurring at irregular time intervals. To address their temporal nature, we treat EHRs as samples generated by a Temporal Point Process (TPP), enabling us to model what happened in an event with when it happened in a principled way. We gather and propose neural network parameterisations of TPPs, collectively referred to as Neural TPPs. We perform evaluations on synthetic EHRs as well as on a set of established benchmarks. We show that TPPs significantly outperform their non-TPP counterparts on EHRs. We also show that an assumption of many Neural TPPs, that the class distribution is conditionally independent of time, reduces performance on EHRs. Finally, our proposed attention-based Neural TPP performs favourably compared to existing models, and provides insight into how it models the EHR, an important step towards a component of clinical decision support systems.

[1]  Vinayak A. Rao,et al.  A Multitask Point Process Predictive Model , 2015, ICML.

[2]  Majid Sarrafzadeh,et al.  Unsupervised Representation for EHR Signals and Codes as Patient Status Vector , 2019, ArXiv.

[3]  Minghe Zhang,et al.  Deep Attention Spatio-Temporal Point Processes , 2020, ArXiv.

[4]  L. Floridi,et al.  The Debate on the Ethics of AI in Health Care: a Reconstruction and Critical Review , 2019, SSRN Electronic Journal.

[5]  Giel Nijpels,et al.  Race/Ethnic Differences in the Associations of the Framingham Risk Factors with Carotid IMT and Cardiovascular Events , 2015, PloS one.

[6]  E. Di Nucci.  Should we be afraid of medical AI? , 2019, Journal of Medical Ethics.

[7]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[8]  L. Floridi,et al.  An ethically mindful approach to AI for health care , 2020, The Lancet.

[9]  Utkarsh Upadhyay,et al.  Recurrent Marked Temporal Point Processes: Embedding Event History to Vector , 2016, KDD.

[10]  Jason Eisner,et al.  The Neural Hawkes Process: A Neurally Self-Modulating Multivariate Point Process , 2016, NIPS.

[11]  Kazuyuki Aihara,et al.  Fully Neural Network based Model for General Temporal Point Processes , 2019, NeurIPS.

[12]  G. Martin,et al.  Inclusive public participation in health: Policy, practice and theoretical contributions to promote the involvement of marginalised groups in healthcare. , 2015, Social science & medicine.

[13]  Brian W. Powers,et al.  Dissecting racial bias in an algorithm used to manage the health of populations , 2019, Science.

[14]  Peter Szolovits,et al.  MIMIC-III, a freely accessible critical care database , 2016, Scientific Data.

[15]  David Page,et al.  Forest-Based Point Process for Event Prediction from Electronic Health Records , 2013, ECML/PKDD.

[16]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[17]  Junqiao Chen,et al.  The validity of synthetic clinical data: a validation study of a leading synthetic data generator (Synthea) using clinical quality measures , 2019, BMC Medical Informatics and Decision Making.

[18]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[19]  Eric J Topol,et al.  High-performance medicine: the convergence of human and artificial intelligence , 2019, Nature Medicine.

[20]  Matthew Le,et al.  Learning Multivariate Hawkes Processes at Scale , 2020, ArXiv.

[21]  J. Overhage,et al.  Physician Time Spent Using the Electronic Health Record During Outpatient Encounters , 2020, Annals of Internal Medicine.

[22]  Christian R. Shelton,et al.  Marked Point Process for Severity of Illness Assessment , 2017, MLHC.

[23]  Stephan Günnemann,et al.  Intensity-Free Learning of Temporal Point Processes , 2020, ICLR.

[24]  Lukasz Kaiser,et al.  Generating Wikipedia by Summarizing Long Sequences , 2018, ICLR.

[25]  W. Herman,et al.  Racial and ethnic differences in the relationship between HbA1c and blood glucose: implications for the diagnosis of diabetes. , 2012, The Journal of clinical endocrinology and metabolism.

[26]  Wei Zhang,et al.  Adverse drug reaction discovery from electronic health records with deep neural networks , 2020, CHIL.

[27]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[28]  Sanja Fidler,et al.  Skip-Thought Vectors , 2015, NIPS.

[29]  M. Graber.  The incidence of diagnostic error in medicine , 2013, BMJ quality & safety.

[30]  Lea Ypi,et al.  Structural Injustice, Epistemic Opacity, and the Responsibilities of the Oppressed , 2019, Journal of Social Philosophy.

[31]  Cynthia Rudin,et al.  Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead , 2018, Nature Machine Intelligence.

[32]  Sihem Amer-Yahia,et al.  Patient trajectory prediction in the Mimic-III dataset, challenges and pitfalls , 2019, ArXiv.

[33]  Luciano Floridi,et al.  The Limits of Empowerment: How to Reframe the Role of mHealth Tools in the Healthcare Ecosystem , 2019, Sci. Eng. Ethics.

[34]  A. Hawkes.  Spectra of some self-exciting and mutually exciting point processes , 1971 .

[35]  M. Komorowski.  Clinical management of sepsis can be improved by artificial intelligence: yes , 2019, Intensive Care Medicine.

[36]  Mark Kramer,et al.  Synthea: An approach, method, and software mechanism for generating synthetic patients and the synthetic electronic health care record , 2017, J. Am. Medical Informatics Assoc..

[37]  Tim Miller,et al.  Explanation in Artificial Intelligence: Insights from the Social Sciences , 2017, Artif. Intell..

[38]  Ulrike Goldschmidt,et al.  An Introduction To The Theory Of Point Processes , 2016 .

[39]  Noah A. Smith,et al.  Is Attention Interpretable? , 2019, ACL.

[40]  É. Kleinpeter.  Four Ethical Issues of “E-Health” , 2017 .

[41]  Mariarosaria Taddeo,et al.  The ethics of algorithms: Mapping the debate , 2016, Big Data Soc..

[42]  Kazem Rahimi,et al.  BEHRT: Transformer for Electronic Health Records , 2019, Scientific Reports.

[43]  Parisa Rashidi,et al.  Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis , 2017, IEEE Journal of Biomedical and Health Informatics.

[44]  Harini Suresh,et al.  A Framework for Understanding Unintended Consequences of Machine Learning , 2019, ArXiv.

[45]  L. Floridi,et al.  Enabling digital health companionship is better than empowerment. , 2019, The Lancet. Digital health.

[46]  Eric E. Smith,et al.  Sex and Race/Ethnicity–Related Disparities in Care and Outcomes After Hospitalization for Coronary Artery Disease Among Older Adults , 2016, Circulation. Cardiovascular quality and outcomes.

[47]  Joseph Sill,et al.  Monotonic Networks , 1997, NIPS.

[48]  Joseph G. Akar,et al.  Aggregating multiple real-world data sources using a patient-centered health-data-sharing platform , 2020, npj Digital Medicine.

[49]  Andrea Lodi,et al.  Activation Adaptation in Neural Networks , 2019, ICPRAM.

[50]  Karen L. Calderone.  The influence of gender on the frequency of pain and sedative medication administered to postoperative patients , 1990 .

[51]  Johannes Gehrke,et al.  Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission , 2015, KDD.

[52]  Felix Hill,et al.  Learning Distributed Representations of Sentences from Unlabelled Data , 2016, NAACL.

[53]  Ben Green,et al.  Disparate Interactions: An Algorithm-in-the-Loop Analysis of Fairness in Risk Assessments , 2019, FAT.

[54]  Tie-Yan Liu,et al.  On Layer Normalization in the Transformer Architecture , 2020, ICML.

[55]  Yoav Goldberg,et al.  Studying the Inductive Biases of RNNs with Synthetic Variations of Natural Languages , 2019, NAACL.

[56]  John Powell,et al.  What is an appropriate level of evidence for a digital health intervention? , 2018, The Lancet.

[57]  Joelle Pineau,et al.  Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program) , 2020, J. Mach. Learn. Res..

[58]  He Zhang,et al.  Electronic health records in IS research: Quality issues, essential thresholds and remedial actions , 2019, Decis. Support Syst..

[59]  S. Kalkman,et al.  Responsible data sharing in international health research: a systematic review of principles and norms , 2019, BMC Medical Ethics.

[60]  Minghe Zhang,et al.  Deep Attention Point Processes with Neural Spectrum Fourier Kernel , 2020 .

[61]  Viju Raghupathi,et al.  An Empirical Study of Chronic Diseases in the United States: A Visual Analytics Approach to Public Health , 2018, International journal of environmental research and public health.

[62]  Christian Biemann,et al.  What do we need to build explainable AI systems for the medical domain? , 2017, ArXiv.

[63]  S. Strakowski,et al.  A Naturalistic Study of Racial Disparities in Diagnoses at an Outpatient Behavioral Health Clinic. , 2019, Psychiatric services.