Joint Medical Ontology Representation Learning for Healthcare Predictions

Healthcare predictions aim at predicting diseases of the next visit to hospital with historical Electronic Health Records (EHR), which is a key research field in personalized healthcare. Previous research has demonstrated that learning meaningful medical ontology representations within the healthcare prediction model can alleviate the data insufficiency problem and thus is beneficial to this task. There are two main pathways of learning medical ontology representations. The first is through pre-defined knowledge graph such as the ICD tree, and the second is through the co-occurrence of diseases within each visit. Majority of existing works formalize their model under only one pathway, and fail to utilize the mutual benefits between them. To exploit these benefits, we propose JMRL, an end-to-end and accurate model for healthcare predictions with Joint Medical ontology Representation Learning. JMRL not only utilizes the joint information from both knowledge graph and co-occurrence statistics, but also make use of the mutual benefits between them in an advanced way with two explicit feedback strategies. Experimental results on the MIMIC-III dataset demonstrate the superiority of our model over all existing state-of-the-art approaches.

[1]  Benjamin C. M. Fung,et al.  Medical Concept Embedding with Multiple Ontological Representations , 2019, IJCAI.

[2]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[3]  Peter Szolovits,et al.  MIMIC-III, a freely accessible critical care database , 2016, Scientific Data.

[4]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[5]  Willie Brink,et al.  Towards Automating Healthcare Question Answering in a Noisy Multilingual Low-Resource Setting , 2019, ACL.

[6]  Fenglong Ma,et al.  Dipole: Diagnosis Prediction in Healthcare via Attention-based Bidirectional Recurrent Neural Networks , 2017, KDD.

[7]  Jimeng Sun,et al.  Multi-layer Representation Learning for Medical Concepts , 2016, KDD.

[8]  Ramakanth Kavuluru,et al.  Few-Shot and Zero-Shot Multi-Label Learning for Structured Label Spaces , 2018, EMNLP.

[9]  Anna Rumshisky,et al.  Unfolding physiological state: mortality modelling in intensive care units , 2014, KDD.

[10]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[11]  Jimeng Sun,et al.  Cloud-based Predictive Modeling System and its Application to Asthma Readmission Prediction , 2015, AMIA.

[12]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[13]  Tomas Mikolov,et al.  Bag of Tricks for Efficient Text Classification , 2016, EACL.

[14]  Le Song,et al.  GRAM: Graph-based Attention Model for Healthcare Representation Learning , 2016, KDD.

[15]  Fenglong Ma,et al.  KAME: Knowledge-based Attention Model for Diagnosis Prediction in Healthcare , 2018, CIKM.

[16]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[17]  Jimeng Sun,et al.  RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism , 2016, NIPS.

[18]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[19]  Charles Elkan,et al.  Learning to Diagnose with LSTM Recurrent Neural Networks , 2015, ICLR.

[20]  Beng Chin Ooi,et al.  Medical Concept Embedding with Time-Aware Attention , 2018, IJCAI.

[21]  Gerhard Weikum,et al.  KnowLife: a versatile approach for constructing a large knowledge graph for biomedical sciences , 2015, BMC Bioinformatics.

[22]  William W. Cohen,et al.  Personalized Recommendations using Knowledge Graphs: A Probabilistic Logic Programming Approach , 2016, RecSys.

[23]  May D. Wang,et al.  Interpretable Predictions of Clinical Outcomes with An Attention-based Recurrent Neural Network , 2017, BCB.