Rethinking domain adaptation for machine learning over clinical language

Abstract Building clinical natural language processing (NLP) systems that work on widely varying data is an absolute necessity because of the expense of obtaining new training data. While domain adaptation research can have a positive impact on this problem, the most widely studied paradigms do not take into account the realities of clinical data sharing. To address this issue, we lay out a taxonomy of domain adaptation, parameterizing by what data is shareable. We show that the most realistic settings for clinical use cases are seriously under-studied. To support research in these important directions, we make a series of recommendations, not just for domain adaptation but for clinical NLP in general, that ensure that data, shared tasks, and released models are broadly useful, and that initiate research directions where the clinical NLP community can lead the broader NLP and machine learning fields.

[1]  James Pustejovsky,et al.  SemEval-2016 Task 12: Clinical TempEval , 2016, NAACL 2016.

[2]  Chen Lin,et al.  Towards generalizable entity-centric clinical coreference resolution , 2017, J. Biomed. Informatics.

[3]  John Blitzer,et al.  Frustratingly Hard Domain Adaptation for Dependency Parsing , 2007, EMNLP.

[4]  Ramesh Raskar,et al.  Distributed learning of deep neural network over multiple agents , 2018, J. Netw. Comput. Appl..

[5]  Christopher D. Manning,et al.  Hierarchical Bayesian Domain Adaptation , 2009, NAACL.

[6]  Roi Reichart,et al.  Neural Structural Correspondence Learning for Domain Adaptation , 2016, CoNLL.

[7]  Yaoyun Zhang,et al.  Adapting Word Embeddings from Multiple Domains to Symptom Recognition from Psychiatric Notes , 2018, AMIA Joint Summits on Translational Science proceedings. AMIA Joint Summits on Translational Science.

[8]  Ruslan Salakhutdinov,et al.  Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks , 2016, ICLR.

[9]  Ken Chen,et al.  Label-Aware Double Transfer Learning for Cross-Specialty Medical Named Entity Recognition , 2018, NAACL.

[10]  Jian Yang,et al.  Instance Selection and Instance Weighting for Cross-Domain Sentiment Classification via PU Learning , 2013, IJCAI.

[11]  Kilian Q. Weinberger,et al.  Marginalized Denoising Autoencoders for Domain Adaptation , 2012, ICML.

[12]  Jianfeng Gao,et al.  Domain Adaptation via Pseudo In-Domain Data Selection , 2011, EMNLP.

[13]  Franck Dernoncourt,et al.  Transfer Learning for Named-Entity Recognition with Neural Networks , 2017, LREC.

[14]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[15]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[16]  James J. Masanz,et al.  Negation’s Not Solved: Generalizability Versus Optimizability in Clinical Natural Language Processing , 2014, PloS one.

[17]  Barbara Plank,et al.  Strong Baselines for Neural Semi-Supervised Learning under Domain Shift , 2018, ACL.

[18]  Hal Daumé,et al.  Frustratingly Easy Domain Adaptation , 2007, ACL.

[19]  Gary D. Bader,et al.  Transfer learning for biomedical named entity recognition with neural networks , 2018, bioRxiv.

[20]  Tatsuya Harada,et al.  Asymmetric Tri-training for Unsupervised Domain Adaptation , 2017, ICML.

[21]  R G Mark,et al.  MIMIC II: a massive temporal ICU patient database to support research in intelligent patient monitoring , 2002, Computers in Cardiology.

[22]  Jingrui He,et al.  Source Free Domain Adaptation Using an Off-the-Shelf Classifier , 2018, 2018 IEEE International Conference on Big Data (Big Data).

[23]  Nanyun Peng,et al.  Multi-task Domain Adaptation for Sequence Tagging , 2016, Rep4NLP@ACL.

[24]  Yoshua Bengio,et al.  Domain Adaptation for Large-Scale Sentiment Classification: A Deep Learning Approach , 2011, ICML.

[25]  Frank Rudzicz,et al.  Multilingual prediction of Alzheimer’s disease through domain adaptation and concept-based language modelling , 2019, NAACL.

[26]  ChengXiang Zhai,et al.  Instance Weighting for Domain Adaptation in NLP , 2007, ACL.

[27]  Timothy A Miller.  Simplified Neural Unsupervised Domain Adaptation , 2019, NAACL-HLT.

[28]  Max Welling,et al.  The Variational Fair Autoencoder , 2015, ICLR.

[29]  Eugene Charniak,et al.  Self-Training for Biomedical Parsing , 2008, ACL.

[30]  John Blitzer,et al.  Domain Adaptation with Structural Correspondence Learning , 2006, EMNLP.

[31]  Young-Bum Kim,et al.  Frustratingly Easy Neural Domain Adaptation , 2016, COLING.

[32]  Sunghwan Sohn,et al.  Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications , 2010, J. Am. Medical Informatics Assoc..

[33]  Guergana K. Savova,et al.  Robust Sentence Segmentation for Clinical Text , 2015, AMIA.

[34]  Lei Zhang,et al.  Transfer Adaptation Learning: A Decade Survey , 2019, IEEE transactions on neural networks and learning systems.