Relaxed Transfer of Different Classes via Spectral Partition

Most existing transfer learning techniques are limited to problems of knowledge transfer across tasks sharing the same set of class labels. In this paper, however, we relax this constraint and propose a spectral-based solution that aims at unveiling the intrinsic structure of the data and generating a partition of the target data, by transferring the eigenspace that well separates the source data. Furthermore, a clustering-based KL divergence is proposed to automatically adjust how much to transfer. We evaluate the proposed model on text and image datasets where class categories of the source and target data are explicitly different, e.g., 3-classes transfer to 2-classes, and show that the proposed approach improves other baselines by an average of 10% in accuracy. The source code and datasets are available from the authors.

[1]  Sergio M. Savaresi,et al.  On the performance of bisecting K-means and PDDP , 2001, SDM.

[2]  Wei Fan,et al.  Actively Transfer Domain Knowledge , 2008, ECML/PKDD.

[3]  Xiang Ji,et al.  Document clustering with prior knowledge , 2006, SIGIR.

[4]  George Karypis,et al.  Evaluation of hierarchical clustering algorithms for document datasets , 2002, CIKM '02.

[5]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Thorsten Joachims,et al.  Transductive Learning via Spectral Graph Partitioning , 2003, ICML.

[7]  Neil D. Lawrence,et al.  Learning to learn with the informative vector machine , 2004, ICML.

[8]  Ming Gu,et al.  Spectral min-max cut for graph partitioning and data clustering , 2001 .

[9]  Ronitt Rubinfeld,et al.  Testing that distributions are close , 2000, Proceedings 41st Annual Symposium on Foundations of Computer Science.

[10]  Rajat Raina,et al.  Constructing informative priors using transfer learning , 2006, ICML.

[11]  Thomas G. Dietterich,et al.  Improving SVM accuracy by training on auxiliary data sources , 2004, ICML.

[12]  Claire Cardie,et al.  Proceedings of the Eighteenth International Conference on Machine Learning, 2001, p. 577–584. Constrained K-means Clustering with Background Knowledge , 2022 .

[13]  Jiawei Han,et al.  Knowledge transfer via multiple model local structure mapping , 2008, KDD.

[14]  Qiang Yang,et al.  Co-clustering based classification for out-of-domain documents , 2007, KDD '07.

[15]  Jianbo Shi,et al.  Grouping with Bias , 2001, NIPS.

[16]  Chris H. Q. Ding,et al.  A min-max cut algorithm for graph partitioning and data clustering , 2001, Proceedings 2001 IEEE International Conference on Data Mining.

[17]  Qiang Yang,et al.  Spectral domain-transfer learning , 2008, KDD.

[18]  Thomas Lengauer,et al.  Multi-task learning for HIV therapy screening , 2008, ICML '08.

[19]  Thomas G. Dietterich,et al.  To transfer or not to transfer , 2005, NIPS 2005.

[20]  Peter A. Flach,et al.  Evaluation Measures for Multi-class Subgroup Discovery , 2009, ECML/PKDD.

[21]  G. Griffin,et al.  Caltech-256 Object Category Dataset , 2007 .

[22]  Gene H. Golub,et al.  Matrix computations , 1983 .

[23]  Koby Crammer,et al.  Analysis of Representations for Domain Adaptation , 2006, NIPS.

[24]  Jeff Bilmes,et al.  A Divergence Prior for Adaptive Learning , 2006 .

[25]  Rich Caruana,et al.  Multitask Learning , 1998, Encyclopedia of Machine Learning and Data Mining.

[26]  Eric Eaton,et al.  Modeling Transfer Relationships Between Learning Tasks for Improved Inductive Transfer , 2008, ECML/PKDD.