Combining Active Learning and Semi-Supervised Learning by Using Selective Label Spreading

In the literature, a number of methods have been proposed for semi-supervised learning. Recently, graph-based methods of semi-supervised learning have become popular because of their capability of handling large amounts of unlabeled data. However, the existing graph based semi-supervised learning algorithms do not optimize the process of selecting better labeled data. We have developed a new selective semi-supervised learning algorithm, called selective label spreading (SLS) by integrating the active learning model into the label spreading framework. SLS optimizes the process of selecting better labeled data in order to improve classification performance. We applied SLS to the well-known hand-written digits recognition data set and demonstrated that SLS can improve the classification performance. The selective label spreading scheme requires a much smaller number of queries to achieve high accuracy compared with random query selection.

[1]  Ellen Riloff,et al.  Learning subjective nouns using extraction pattern bootstrapping , 2003, CoNLL.

[2]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[3]  Sebastian Thrun,et al.  Text Classification from Labeled and Unlabeled Documents using EM , 2000, Machine Learning.

[4]  Pascal Vincent,et al.  The Manifold Tangent Classifier , 2011, NIPS.

[5]  Xiaojin Zhu,et al.  Harmonic mixtures: combining mixture models and graph-based methods for inductive and scalable semi-supervised learning , 2005, ICML.

[6]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[7]  Raymond J. Mooney,et al.  Active Learning for Probability Estimation Using Jensen-Shannon Divergence , 2005, ECML.

[8]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[9]  Patrick Fox-Roberts,et al.  Unbiased generative semi-supervised learning , 2014, J. Mach. Learn. Res..

[10]  Zhang Chi,et al.  SOAL: Second-Order Online Active Learning , 2016 .

[11]  Zhao Hong-hai.  Semi-Supervised Support Vector Machines for Data Classification , 2004 .

[12]  David Yarowsky,et al.  Rule Writing or Annotation: Cost-efficient Resource Usage for Base Noun Phrase Chunking , 2000, ACL.

[13]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[14]  Fredrik Olsson,et al.  A literature survey of active machine learning in the context of natural language processing , 2009 .

[15]  Marc'Aurelio Ranzato,et al.  Semi-supervised learning of compact document representations with deep networks , 2008, ICML '08.

[16]  Andrew McCallum,et al.  Employing EM and Pool-Based Active Learning for Text Classification , 1998, ICML.

[17]  Mark Craven,et al.  An Analysis of Active Learning Strategies for Sequence Labeling Tasks , 2008, EMNLP.

[18]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[19]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[20]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[21]  Fabio Gagliardi Cozman,et al.  Semi-Supervised Learning of Mixture Models , 2003, ICML.

[22]  Dale Schuurmans,et al.  Discriminative Batch Mode Active Learning , 2007, NIPS.

[23]  Alexander Zien,et al.  Semi-Supervised Learning , 2006 .

[24]  Ayhan Demiriz,et al.  Semi-Supervised Support Vector Machines , 1998, NIPS.

[25]  James T. Kwok,et al.  Prototype vector machine for large scale semi-supervised learning , 2009, ICML '09.

[26]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[27]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[28]  Shumeet Baluja,et al.  Probabilistic Modeling for Face Orientation Discrimination: Learning from Labeled and Unlabeled Data , 1998, NIPS.

[29]  Martial Hebert,et al.  Semi-Supervised Self-Training of Object Detection Models , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[30]  David Yarowsky,et al.  Unsupervised Word Sense Disambiguation Rivaling Supervised Methods , 1995, ACL.

[31]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[32]  Xiaojin Zhu,et al.  Semi-Supervised Learning , 2010, Encyclopedia of Machine Learning.

[33]  Tom Michael Mitchell,et al.  The Role of Unlabeled Data in Supervised Learning , 2004 .

[34]  Antonio Torralba,et al.  Semi-Supervised Learning in Gigantic Image Collections , 2009, NIPS.

[35]  Chen Huang,et al.  Reliable Semi-supervised Learning , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[36]  Naoki Abe,et al.  Query Learning Strategies Using Boosting and Bagging , 1998, ICML.

[37]  Hossein Mobahi,et al.  Deep Learning via Semi-supervised Embedding , 2012, Neural Networks: Tricks of the Trade.

[38]  David D. Lewis,et al.  Heterogeneous Uncertainty Sampling for Supervised Learning , 1994, ICML.

[39]  William A. Gale,et al.  A sequential algorithm for training text classifiers , 1994, SIGIR '94.

[40]  J. Lafferty,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[41]  Xiaojin Zhu,et al.  --1 CONTENTS , 2006 .

[42]  Shlomo Argamon,et al.  Minimizing Manual Annotation Cost in Supervised Training from Corpora , 1996, ACL.

[43]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[44]  Vittorio Castelli,et al.  On the exponential value of labeled samples , 1995, Pattern Recognit. Lett..

[45]  Neil D. Lawrence,et al.  Semi-supervised Learning via Gaussian Processes , 2004, NIPS.

[46]  Shlomo Argamon,et al.  Committee-Based Sampling For Training Probabilistic Classi(cid:12)ers , 1995 .

[47]  H. Sebastian Seung,et al.  Query by committee , 1992, COLT '92.