Softly Associative Transfer Learning for Cross-Domain Classification

The main challenge of cross-domain text classification is to train a classifier in a source domain while applying it to a different target domain. Many transfer learning-based algorithms, for example, dual transfer learning, triplex transfer learning, etc., have been proposed for cross-domain classification, by detecting a shared low-dimensional feature representation for both source and target domains. These methods, however, often assume that the word clusters matrix or the clusters association matrix as knowledge transferring bridges are exactly the same across different domains, which is actually unrealistic in real-world applications and, therefore, could degrade classification performance. In light of this, in this paper, we propose a softly associative transfer learning algorithm for cross-domain text classification. Specifically, we integrate two non-negative matrix tri-factorizations into a joint optimization framework, with approximate constraints on both word clusters matrices and clusters association matrices so as to allow proper diversity in knowledge transfer, and with another approximate constraint on class labels in source domains in order to handle noisy labels. An iterative algorithm is then proposed to solve the above problem, with its convergence verified theoretically and empirically. Extensive experimental results on various text datasets demonstrate the effectiveness of our algorithm, even with the presence of abundant state-of-the-art competitors.

[1]  Ming Shao,et al.  Incomplete Multisource Transfer Learning , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[2]  Qiang Yang,et al.  Transfer Learning by Structural Analogy , 2011, AAAI.

[3]  Qiang Yang,et al.  Co-clustering based classification for out-of-domain documents , 2007, KDD '07.

[4]  Hui Xiong,et al.  Exploiting Associations between Word Clusters and Document Classes for Cross-Domain Text Categorization , 2010, SDM.

[5]  Fuzhen Zhuang,et al.  Triplex Transfer Learning: Exploiting Both Shared and Distinct Concepts for Text Classification , 2013, IEEE Transactions on Cybernetics.

[6]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[7]  Hui Xiong,et al.  K-Means-Based Consensus Clustering: A Unified View , 2015, IEEE Transactions on Knowledge and Data Engineering.

[8]  Ivor W. Tsang,et al.  Transfer Learning for Cross-Language Text Categorization through Active Correspondences Construction , 2016, AAAI.

[9]  Zhongfei Zhang,et al.  Structural Correspondence Learning for Cross-Lingual Sentiment Classification with One-to-Many Mappings , 2016, AAAI.

[10]  Benno Stein,et al.  Cross-Language Text Classification Using Structural Correspondence Learning , 2010, ACL.

[11]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[12]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[13]  Fuzhen Zhuang,et al.  Concept Learning for Cross-Domain Text Classification: A General Probabilistic Framework , 2013, IJCAI.

[14]  Ke Lu,et al.  Transfer Independently Together: A Generalized Framework for Domain Adaptation , 2019, IEEE Transactions on Cybernetics.

[15]  Benno Stein,et al.  Cross-Lingual Adaptation Using Structural Correspondence Learning , 2010, TIST.

[16]  Jianmin Wang,et al.  Transfer Learning with Graph Co-Regularization , 2012, IEEE Transactions on Knowledge and Data Engineering.

[17]  Tingting He,et al.  A Subspace Learning Framework for Cross-Lingual Sentiment Classification with Partial Parallel Data , 2015, IJCAI.

[18]  Seungjin Choi,et al.  Orthogonal nonnegative matrix tri-factorization for co-clustering: Multiplicative updates on Stiefel manifolds , 2010, Inf. Process. Manag..

[19]  Chris H. Q. Ding,et al.  Knowledge transformation for cross-domain sentiment classification , 2009, SIGIR.

[20]  Ke Lu,et al.  Heterogeneous Domain Adaptation Through Progressive Alignment , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[21]  Min Jiang,et al.  Integration of Global and Local Metrics for Domain Adaptation Learning Via Dimensionality Reduction , 2017, IEEE Transactions on Cybernetics.

[22]  Chris H. Q. Ding,et al.  Orthogonal nonnegative matrix t-factorizations for clustering , 2006, KDD '06.

[23]  Xiaolong Wang,et al.  Cross-lingual Opinion Analysis via Negative Transfer Detection , 2014, ACL.

[24]  Jiawei Han,et al.  Knowledge transfer via multiple model local structure mapping , 2008, KDD.

[25]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[26]  John Blitzer,et al.  Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification , 2007, ACL.

[27]  Qiang Yang,et al.  Cross-domain sentiment classification via spectral feature alignment , 2010, WWW '10.

[28]  Yong Yu,et al.  Cross-Lingual Sentiment Classification via Bi-view Non-negative Matrix Tri-Factorization , 2011, PAKDD.

[29]  Ming Shao,et al.  Structure-Preserved Multi-source Domain Adaptation , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[30]  Ke Lu,et al.  Low-Rank Discriminant Embedding for Multiview Learning , 2017, IEEE Transactions on Cybernetics.

[31]  Dan Zhang,et al.  Multi-view transfer learning with a large margin approach , 2011, KDD.

[32]  Yuan Yan Tang,et al.  Cross-Domain Recognition by Identifying Joint Subspaces of Source Domain and Target Domain , 2017, IEEE Transactions on Cybernetics.

[33]  Guodong Zhou,et al.  Active Learning for Cross-domain Sentiment Classification , 2013, IJCAI.

[34]  Houfeng Wang,et al.  Cross-Lingual Mixture Model for Sentiment Classification , 2012, ACL.

[35]  Feiping Nie,et al.  Cross-language web page classification via dual knowledge transfer using nonnegative matrix tri-factorization , 2011, SIGIR.

[36]  Qiang Yang,et al.  Transitive Transfer Learning , 2015, KDD.

[37]  Thomas Hofmann,et al.  Unsupervised Learning by Probabilistic Latent Semantic Analysis , 2004, Machine Learning.

[38]  Boris G. Mirkin,et al.  Reinterpreting the Category Utility Function , 2001, Machine Learning.

[39]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[40]  Massimiliano Pontil,et al.  Regularized multi--task learning , 2004, KDD.

[41]  Yuhong Zhang,et al.  Multi-bridge transfer learning , 2016, Knowl. Based Syst..

[42]  Jianmin Wang,et al.  Dual Transfer Learning , 2012, SDM.

[43]  Dacheng Tao,et al.  Classification with Noisy Labels by Importance Reweighting , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.