Multi-directional Recurrent Neural Networks : A Novel Method for Estimating Missing Data

Most time-series datasets with multiple data streams have (many) missing measurements that need to be estimated. Most existing methods address this estimation problem either by interpolating within data streams or imputing across data streams; we develop a novel approach that does both. Our approach is based on a deep learning architecture that we call a Multidirectional Recurrent Neural Network (M-RNN). An M-RNN differs from a bi-directional RNN in that it operates across streams in addition to within streams, and because the timing of inputs into the hidden layers is both lagged and advanced. To demonstrate the power of our approach we apply it to a familiar real-world medical dataset and demonstrate significantly improved performance.

[1]  Yoshua Bengio,et al.  Recurrent Neural Networks for Missing or Asynchronous Data , 1995, NIPS.

[2]  Volker Tresp,et al.  A Solution for Missing Data in Recurrent Neural Networks with an Application to Blood Glucose Prediction , 1997, NIPS.

[3]  Phil D. Green,et al.  Speech Recognition with Missing Data using Recurrent Neural Nets , 2001, NIPS.

[4]  Jürgen Schmidhuber,et al.  Framewise phoneme classification with bidirectional LSTM and other neural network architectures , 2005, Neural Networks.

[5]  David M Kreindler,et al.  The effects of the irregular sample and missing data in time series analysis. , 2006, Nonlinear dynamics, psychology, and life sciences.

[6]  Aníbal R. Figueiras-Vidal,et al.  Pattern classification with missing data: a review , 2010, Neural Computing and Applications.

[7]  D. Percival,et al.  Wavelet variance analysis for gappy time series , 2010 .

[8]  J. Kurths,et al.  Comparison of correlation analysis techniques for irregularly sampled time series , 2011 .

[9]  Patrick Royston,et al.  Multiple imputation using chained equations: Issues and guidance for practice , 2011, Statistics in medicine.

[10]  Inderjit S. Dhillon,et al.  Temporal Regularized Matrix Factorization , 2015, ArXiv.

[11]  David C. Kale,et al.  Directly Modeling Missing Data in Sequences with RNNs: Improved Classification of Clinical Time Series , 2016, MLHC.

[12]  Peter Szolovits,et al.  MIMIC-III, a freely accessible critical care database , 2016, Scientific Data.

[13]  Walter F. Stewart,et al.  Doctor AI: Predicting Clinical Events via Recurrent Neural Networks , 2015, MLHC.

[14]  Thorsten Joachims,et al.  Recommendations as Treatments: Debiasing Learning and Evaluation , 2016, ICML.

[15]  Mihaela van der Schaar,et al.  Learning from Clinical Judgments: Semi-Markov-Modulated Marked Hawkes Processes for Risk Prognosis , 2017, ICML.

[16]  Ho-Jin Choi,et al.  Recurrent neural networks with missing information imputation for medical examination data prediction , 2017, 2017 IEEE International Conference on Big Data and Smart Computing (BigComp).

[17]  Yan Liu,et al.  Recurrent Neural Networks for Multivariate Time Series with Missing Values , 2016, Scientific Reports.