Hierarchical Active Transfer Learning

We describe a unified active transfer learning framework called Hierarchical Active Transfer Learning (HATL). HATL exploits cluster structure shared between different data domains to perform transfer learning by imputing labels for unlabeled target data and to generate effective label queries during active learning. The resulting framework is flexible enough to perform not only adaptive transfer learning and accelerated active learning but also unsupervised and semi-supervised transfer learning. We derive an intuitive and useful upper bound on HATL’s error when used to infer labels for unlabeled target points. We also present results on synthetic data that confirm both intuition and our analysis. Finally, we demonstrate HATL’s empirical effectiveness on a benchmark data set for sentiment classification.

[1]  Burr Settles,et al.  Active Learning , 2012, Synthesis Lectures on Artificial Intelligence and Machine Learning.

[2]  Sanjoy Dasgupta,et al.  Hierarchical sampling for active learning , 2008, ICML '08.

[3]  Xinyu Dai,et al.  Active Learning with Transfer Learning , 2012, ACL 2012.

[4]  Avishek Saha,et al.  Co-regularization Based Semi-supervised Domain Adaptation , 2010, NIPS.

[5]  Jaime G. Carbonell,et al.  Identifiability of Priors from Bounded Sample Sizes with Applications to Transfer Learning , 2011, COLT.

[6]  Deborah Estrin,et al.  Improving activity classification for health applications on mobile devices using active and semi-supervised learning , 2010, 2010 4th International Conference on Pervasive Computing Technologies for Healthcare.

[7]  Koby Crammer,et al.  Learning Bounds for Domain Adaptation , 2007, NIPS.

[8]  Rajat Raina,et al.  Self-taught learning: transfer learning from unlabeled data , 2007, ICML '07.

[9]  Daumé,et al.  Domain Adaptation meets Active Learning , 2010, HLT-NAACL 2010.

[10]  Rasoul Karimi,et al.  Active Learning for Recommender Systems , 2015, KI - Künstliche Intelligenz.

[11]  David Haussler,et al.  Quantifying Inductive Bias: AI Learning Algorithms and Valiant's Learning Framework , 1988, Artif. Intell..

[12]  Hua Xu,et al.  Applying active learning to high-throughput phenotyping algorithms for electronic health records data. , 2013, Journal of the American Medical Informatics Association : JAMIA.

[13]  Bernhard Schölkopf,et al.  A Kernel Method for the Two-Sample-Problem , 2006, NIPS.

[14]  David R. Thompson,et al.  Semi-Supervised Novelty Detection with Adaptive Eigenbases, and Application to Radio Transients , 2011 .

[15]  Avishek Saha,et al.  Active Supervised Domain Adaptation , 2011, ECML/PKDD.

[16]  Wei Fan,et al.  Actively Transfer Domain Knowledge , 2008, ECML/PKDD.

[17]  Jon D. Patrick,et al.  Research and applications: Supervised machine learning and active learning in classification of radiology reports , 2014, J. Am. Medical Informatics Assoc..

[18]  Sethuraman Panchanathan,et al.  Joint Transfer and Batch-mode Active Learning , 2013, ICML.

[19]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[20]  Sanjoy Dasgupta,et al.  Two faces of active learning , 2011, Theor. Comput. Sci..

[21]  John Langford,et al.  Agnostic Active Learning Without Constraints , 2010, NIPS.

[22]  Shai Ben-David,et al.  Detecting Change in Data Streams , 2004, VLDB.

[23]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[24]  John Blitzer,et al.  Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification , 2007, ACL.

[25]  Yan Liu,et al.  Accelerating Active Learning with Transfer Learning , 2013, 2013 IEEE 13th International Conference on Data Mining.

[26]  Jaime G. Carbonell,et al.  A theory of transfer learning with applications to active learning , 2013, Machine Learning.

[27]  Gunnar Rätsch,et al.  Active Learning in the Drug Discovery Process , 2001, NIPS.