Variational Recurrent Adversarial Deep Domain Adaptation

We study the problem of learning domain invariant representations for time series data while transferring the complex temporal latent dependencies between domains. Our model termed as Variational Recurrent Adversarial Deep Domain Adaptation (VRADA) is built atop a variational recurrent neural network (VRNN) and trains adversarially to capture complex temporal relationships that are domain-invariant. This is (as far as we know) the first to capture and transfer temporal latent dependencies of multivariate time-series data. Through experiments on real-world multivariate healthcare time-series datasets, we empirically demonstrate that learning temporal dependencies helps our model’s ability to create domain-invariant representations, allowing our model to outperform current state-of-the-art deep domain adaptation approaches.

[1]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[2]  Meena Seshamani,et al.  A longitudinal study of the effects of age and time to death on hospital costs. , 2004, Journal of health economics.

[3]  Yoshua Bengio,et al.  A Recurrent Latent Variable Model for Sequential Data , 2015, NIPS.

[4]  K. Warner,et al.  The lifetime distribution of health care costs. , 2004, Health services research.

[5]  Yuan Shi,et al.  Geodesic flow kernel for unsupervised domain adaptation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Michael I. Jordan,et al.  Learning Transferable Features with Deep Adaptation Networks , 2015, ICML.

[7]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[8]  Rama Chellappa,et al.  Visual Domain Adaptation: A survey of recent advances , 2015, IEEE Signal Processing Magazine.

[9]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[10]  Koby Crammer,et al.  A theory of learning from different domains , 2010, Machine Learning.

[11]  R. Zemel,et al.  THE VARIATIONAL FAIR AUTO ENCODER , 2015 .

[12]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[13]  Alexander Yates,et al.  Distributional Representations for Handling Sparsity in Supervised Sequence-Labeling , 2009, ACL.

[14]  Dinggang Shen,et al.  Morphological classification of brains via high-dimensional shape transformations and machine learning methods , 2004, NeuroImage.

[15]  James J. Jiang.  A Literature Survey on Domain Adaptation of Statistical Classifiers , 2007 .

[16]  Andrew Y. Ng,et al.  Parsing Natural Scenes and Natural Language with Recursive Neural Networks , 2011, ICML.

[17]  Kilian Q. Weinberger,et al.  Marginalized Denoising Autoencoders for Domain Adaptation , 2012, ICML.

[18]  Koby Crammer,et al.  Analysis of Representations for Domain Adaptation , 2006, NIPS.

[19]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[20]  ChengXiang Zhai,et al.  Instance Weighting for Domain Adaptation in NLP , 2007, ACL.

[21]  Victor S. Lempitsky,et al.  Unsupervised Domain Adaptation by Backpropagation , 2014, ICML.

[22]  John Blitzer,et al.  Domain adaptation of natural language processing systems , 2008 .

[23]  Tinne Tuytelaars,et al.  Unsupervised Visual Domain Adaptation Using Subspace Alignment , 2013, 2013 IEEE International Conference on Computer Vision.

[24]  Peter Szolovits,et al.  MIMIC-III, a freely accessible critical care database , 2016, Scientific Data.

[25]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[26]  Roland Kuhn,et al.  Discriminative Instance Weighting for Domain Adaptation in Statistical Machine Translation , 2010, EMNLP.

[27]  Trevor Darrell,et al.  Simultaneous Deep Transfer Across Domains and Tasks , 2015, ICCV.

[28]  T. Alonzo,et al.  Effect of tidal volume in children with acute hypoxemic respiratory failure , 2009, Intensive Care Medicine.

[29]  Trevor Darrell,et al.  Adapting Visual Category Models to New Domains , 2010, ECCV.