Domain Adaptation for Sentiment Analysis Using Increased Intraclass Separation

Sentiment analysis is a costly yet necessary task for enterprises to study the opinions of their customers to improve their products and to determine optimal marketing strategies. Due to the existence of a wide range of domains across different products and services, cross-domain sentiment analysis methods have received significant attention. These methods mitigate the domain gap between different applications by training cross-domain generalizable classifiers which help to relax the need for data annotation for each domain. Most existing methods focus on learning domain-agnostic representations that are invariant with respect to both the source and the target domains. As a result, a classifier that is trained using the source domain annotated data would generalize well in a related target domain. We introduce a new domain adaptation method which induces large margins between different classes in an embedding space. This embedding space is trained to be domain-agnostic by matching the data distributions across the domains. Large intraclass margins in the source domain help to reduce the effect of “domain shift” on the classifier performance in the target domain. Theoretical and empirical analysis are provided to demonstrate that the proposed method is effective.

[1]  Leland McInnes,et al.  UMAP: Uniform Manifold Approximation and Projection , 2018, J. Open Source Softw..

[2]  Chen-Yu Lee,et al.  Sliced Wasserstein Discrepancy for Unsupervised Domain Adaptation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Nicolas Bonnotte.  Unidimensional and Evolution Methods for Optimal Transportation , 2013 .

[4]  Barbara Caputo,et al.  Frustratingly Easy NBNN Domain Adaptation , 2013, 2013 IEEE International Conference on Computer Vision.

[5]  Tatsuya Harada,et al.  Asymmetric Tri-training for Unsupervised Domain Adaptation , 2017, ICML.

[6]  Charles Elkan,et al.  Expectation Maximization Algorithm , 2010, Encyclopedia of Machine Learning.

[7]  Tat-Seng Chua,et al.  Multi-source Domain Adaptation for Visual Sentiment Classification , 2020, AAAI.

[8]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[9]  Nicolas Courty,et al.  DeepJDOT: Deep Joint distribution optimal transport for unsupervised domain adaptation , 2018, ECCV.

[10]  Fuzhen Zhuang,et al.  Domain Adaptation with Category Attention Network for Deep Sentiment Analysis , 2020, WWW.

[11]  Mohammad Rostami,et al.  Generative Continual Concept Learning , 2019, AAAI.

[12]  Mohammad Rostami,et al.  Learning Transferable Knowledge Through Embedding Spaces , 2019 .

[13]  Michael I. Jordan,et al.  Transferable Adversarial Training: A General Approach to Adapting Deep Classifiers , 2019, ICML.

[14]  Nicolas Courty,et al.  Optimal Transport for Domain Adaptation , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Michael I. Jordan,et al.  Learning Transferable Features with Deep Adaptation Networks , 2015, ICML.

[16]  Michael I. Jordan,et al.  Conditional Adversarial Domain Adaptation , 2017, NeurIPS.

[17]  Qiang Yang,et al.  Transferable End-to-End Aspect-based Sentiment Analysis with Selective Adversarial Learning , 2019, EMNLP.

[18]  Kate Saenko,et al.  Deep CORAL: Correlation Alignment for Deep Domain Adaptation , 2016, ECCV Workshops.

[19]  Barbara Plank,et al.  Strong Baselines for Neural Semi-Supervised Learning under Domain Shift , 2018, ACL.

[20]  Jianfei Yu,et al.  Unified Feature and Instance Based Domain Adaptation for End-to-End Aspect-based Sentiment Analysis , 2020, EMNLP.

[21]  Shai Ben-David,et al.  Understanding Machine Learning: From Theory to Algorithms , 2014 .

[22]  Xuanjing Huang,et al.  Cross-Domain Sentiment Classification with Target Domain Specific Information , 2018, ACL.

[23]  Ievgen Redko,et al.  Theoretical Analysis of Domain Adaptation with Optimal Transport , 2016, ECML/PKDD.

[24]  Mohammad Rostami,et al.  A crowdsourcing triage algorithm for geopolitical event forecasting , 2018, RecSys.

[25]  Roi Reichart,et al.  Task Refinement Learning for Improved Accuracy and Stability of Unsupervised Domain Adaptation , 2019, ACL.

[26]  Mohammad Rostami,et al.  Privacy Preserving Domain Adaptation for Semantic Segmentation of Medical Images , 2021, ArXiv.

[27]  Mei Wang,et al.  Deep Visual Domain Adaptation: A Survey , 2018, Neurocomputing.

[28]  Longhui Wei,et al.  Person Transfer GAN to Bridge Domain Gap for Person Re-identification , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[29]  Sam T. Roweis,et al.  EM Algorithms for PCA and SPCA , 1997, NIPS.

[30]  Mohammad Rostami,et al.  Sequential Unsupervised Domain Adaptation through Prototypical Distributions , 2020, ArXiv.

[31]  Mohammad Rostami,et al.  Complementary Learning for Overcoming Catastrophic Forgetting Using Experience Replay , 2019, IJCAI.

[32]  Roi Reichart,et al.  Pivot Based Language Modeling for Improved Neural Domain Adaptation , 2018, NAACL.

[33]  Jeremy Barnes,et al.  Projecting Embeddings for Domain Adaption: Joint Modeling of Sentiment Analysis in Diverse Domains , 2018, COLING.

[34]  Jingyu Wang,et al.  Adversarial and Domain-Aware BERT for Cross-Domain Sentiment Analysis , 2020, ACL.

[35]  George Trigeorgis,et al.  Domain Separation Networks , 2016, NIPS.

[36]  Jianmin Wang,et al.  Multi-Adversarial Domain Adaptation , 2018, AAAI.

[37]  Edwin Lughofer,et al.  Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learning , 2017, ICLR.

[38]  Jian Shen,et al.  Wasserstein Distance Guided Representation Learning for Domain Adaptation , 2017, AAAI.

[39]  Yi Yang,et al.  Contrastive Adaptation Network for Unsupervised Domain Adaptation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  C. Villani,et al.  Quantitative Concentration Inequalities for Empirical Measures on Non-compact Spaces , 2005, math/0503123.

[41]  Yingyu Liang,et al.  Shallow Domain Adaptive Embeddings for Sentiment Analysis , 2019, EMNLP/IJCNLP.

[42]  Zenglin Xu,et al.  Adversarial Training Based Multi-Source Unsupervised Domain Adaptation for Sentiment Analysis , 2020, AAAI.

[43]  Mohammad Rostami,et al.  Unsupervised Model Adaptation for Continual Semantic Segmentation , 2020, AAAI.

[44]  John Blitzer,et al.  Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification , 2007, ACL.

[45]  Han Guo.  Multi-Source Domain Adaptation for Text Classification via DistanceNet-Bandits , 2020 .

[46]  Fangzhao Wu,et al.  Sentiment Domain Adaptation with Multiple Sources , 2016, ACL.

[47]  Eric Eaton,et al.  Deep Transfer Learning for Few-Shot SAR Image Classification , 2019, Remote. Sens..