Self-Supervised Graph Learning With Hyperbolic Embedding for Temporal Health Event Prediction

Electronic health records (EHRs) have been heavily used in modern healthcare systems for recording patients' admission information to health facilities. Many data-driven approaches employ temporal features in EHR for predicting specific diseases, readmission times, and diagnoses of patients. However, most existing predictive models cannot fully utilize EHR data, due to an inherent lack of labels in supervised training for some temporal events. Moreover, it is hard for the existing methods to simultaneously provide generic and personalized interpretability. To address these challenges, we propose Sherbet, a self-supervised graph learning framework with hyperbolic embeddings for temporal health event prediction. We first propose a hyperbolic embedding method with information flow to pretrain medical code representations in a hierarchical structure. We incorporate these pretrained representations into a graph neural network (GNN) to detect disease complications and design a multilevel attention method to compute the contributions of particular diseases and admissions, thus enhancing personalized interpretability. We present a new hierarchy-enhanced historical prediction proxy task in our self-supervised learning framework to fully utilize EHR data and exploit medical domain knowledge. We conduct a comprehensive set of experiments on widely used publicly available EHR datasets to verify the effectiveness of our model. Our results demonstrate the proposed model's strengths in both predictive tasks and interpretable abilities.

[1]  S. Bangalore,et al.  The Transition From Hypertension to Heart Failure: Contemporary Update. , 2017, JACC. Heart failure.

[2]  Shanshan Zhang,et al.  Interpretable Representation Learning for Healthcare via Capturing Disease Progression through Time , 2018, KDD.

[3]  Peter Szolovits,et al.  MIMIC-III, a freely accessible critical care database , 2016, Scientific Data.

[4]  Alistair E. W. Johnson,et al.  The eICU Collaborative Research Database, a freely available multi-center database for critical care research , 2018, Scientific Data.

[5]  Chandan K. Reddy,et al.  Collaborative Graph Learning with Auxiliary Text for Temporal Event Prediction in Healthcare , 2021, IJCAI.

[6]  Bin Jiang,et al.  A Data-Driven Aero-Engine Degradation Prognostic Strategy , 2019, IEEE Transactions on Cybernetics.

[7]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[8]  Baoyao Yang,et al.  Cross-Domain Missingness-Aware Time-Series Adaptation With Similarity Distillation in Medical Applications , 2020, IEEE Transactions on Cybernetics.

[9]  Yujia Li,et al.  Learning the Graphical Structure of Electronic Health Records with Graph Convolutional Transformer , 2020, AAAI.

[10]  Fenglong Ma,et al.  Dipole: Diagnosis Prediction in Healthcare via Attention-based Bidirectional Recurrent Neural Networks , 2017, KDD.

[11]  Majid Sarrafzadeh,et al.  TAPER: Time-Aware Patient EHR Representation , 2019, IEEE Journal of Biomedical and Health Informatics.

[12]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[13]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[14]  Fenglong Ma,et al.  HiTANet: Hierarchical Time-Aware Attention Networks for Risk Prediction on Electronic Health Records , 2020, KDD.

[15]  Baogang Wei,et al.  A Topic Modeling Approach for Traditional Chinese Medicine Prescriptions , 2018, IEEE Transactions on Knowledge and Data Engineering.

[16]  Nilmini Wickramasinghe,et al.  Deepr: A Convolutional Net for Medical Records , 2016, ArXiv.

[17]  Heung-Il Suk,et al.  Uncertainty-Aware Variational-Recurrent Imputation Network for Clinical Time Series , 2019, IEEE Transactions on Cybernetics.

[18]  Yu Cheng,et al.  Boosting Deep Learning Risk Prediction with Generative Adversarial Networks for Electronic Health Records , 2017, 2017 IEEE International Conference on Data Mining (ICDM).

[19]  Chandan K. Reddy,et al.  Self-Supervised Hyperboloid Representations from Logical Queries over Knowledge Graphs , 2021, WWW.

[20]  Jimeng Sun,et al.  Pre-training of Graph Augmented Transformers for Medication Recommendation , 2019, IJCAI.

[21]  Jingmin Xin,et al.  Predicting COVID-19 in China Using Hybrid AI Model , 2020, IEEE Transactions on Cybernetics.

[22]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[23]  Le Song,et al.  GRAM: Graph-based Attention Model for Healthcare Representation Learning , 2016, KDD.

[24]  Yingli Tian,et al.  Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[26]  Qi Xuan,et al.  A Comorbidity Knowledge-Aware Model for Disease Prognostic Prediction , 2021, IEEE Transactions on Cybernetics.

[27]  Liang Chen,et al.  Self-supervised learning for medical image analysis using image context restoration , 2019, Medical Image Anal..

[28]  Huimin Lu,et al.  Ternary Adversarial Networks With Self-Supervision for Zero-Shot Cross-Modal Retrieval , 2020, IEEE Transactions on Cybernetics.

[29]  Huilong Duan,et al.  A Regularized Deep Learning Approach for Clinical Risk Prediction of Acute Coronary Syndrome Using Electronic Health Records , 2018, IEEE Transactions on Biomedical Engineering.

[30]  Andrew M. Dai,et al.  Embedding Text in Hyperbolic Spaces , 2018, TextGraphs@NAACL-HLT.

[31]  Douwe Kiela,et al.  Poincaré Embeddings for Learning Hierarchical Representations , 2017, NIPS.

[32]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[33]  Wei Luo,et al.  Effective Identification of Similar Patients Through Sequential Matching over ICD Code Embedding , 2018, Journal of Medical Systems.

[34]  Philip S. Yu,et al.  Mixed Pooling Multi-View Attention Autoencoder for Representation Learning in Healthcare , 2019, ArXiv.

[35]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[36]  Christopher D. Manning,et al.  Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.

[37]  Jimeng Sun,et al.  RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism , 2016, NIPS.

[38]  Fei Wang,et al.  Patient Subtyping via Time-Aware LSTM Networks , 2017, KDD.

[39]  Christopher R'e,et al.  Low-Dimensional Hyperbolic Knowledge Graph Embeddings , 2020, ACL.

[40]  Medicaid Services,et al.  International Classification of Diseases, Ninth Revision, Clinical Modification , 2011 .

[41]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[42]  Nikos Komodakis,et al.  Unsupervised Representation Learning by Predicting Image Rotations , 2018, ICLR.

[43]  Svetha Venkatesh,et al.  $\mathtt {Deepr}$: A Convolutional Net for Medical Records , 2016, IEEE Journal of Biomedical and Health Informatics.

[44]  Jimeng Sun,et al.  MiME: Multilevel Medical Embedding of Electronic Health Records for Predictive Healthcare , 2018, NeurIPS.

[45]  Walter F. Stewart,et al.  Doctor AI: Predicting Clinical Events via Recurrent Neural Networks , 2015, MLHC.

[46]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[47]  Tianqi Chen,et al.  Empirical Evaluation of Rectified Activations in Convolutional Network , 2015, ArXiv.

[48]  Meng Wang,et al.  Adaptive Hypergraph Learning and its Application in Image Classification , 2012, IEEE Transactions on Image Processing.

[49]  V. Ovcharov [International classification of diseases (tenth revision)]. , 1998, Problemy sotsial'noi gigieny i istoriia meditsiny.