Muffled Semi-Supervised Learning

We explore a novel approach to semi-supervised learning. This approach is contrary to the common approach in that the unlabeled examples serve to "muffle," rather than enhance, the guidance provided by the labeled examples. We provide several variants of the basic algorithm and show experimentally that they can achieve significantly higher AUC than boosted trees, random forests and logistic regression when unlabeled examples are available.

[1]  L. Brown,et al.  Interval Estimation for a Binomial Proportion , 2001 .

[2]  Yoav Freund,et al.  Optimal Binary Classifier Aggregation for General Losses , 2015, NIPS.

[3]  Jared Nambwenya,et al.  Give Me Some Credit , 2014 .

[4]  Alexander J. Smola,et al.  Estimating labels from label proportions , 2008, ICML '08.

[5]  Xiaojin Zhu,et al.  Semi-Supervised Learning , 2010, Encyclopedia of Machine Learning.

[6]  Yoav Freund,et al.  Optimally Combining Classifiers Using Unlabeled Data , 2015, COLT.

[7]  Yoav Freund,et al.  The Alternating Decision Tree Learning Algorithm , 1999, ICML.

[8]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[9]  Alex M. Andrew,et al.  Boosting: Foundations and Algorithms , 2012 .

[10]  Yoav Freund,et al.  Scalable Semi-Supervised Classifier Aggregation , 2015, ArXiv.

[11]  Yi Liu,et al.  SemiBoost: Boosting for Semi-Supervised Learning , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[13]  Gunnar Rätsch,et al.  Totally corrective boosting algorithms that maximize the margin , 2006, ICML.

[14]  J. Kiefer,et al.  Sequential minimax search for a maximum , 1953 .

[15]  E. B. Wilson.  Probable Inference, the Law of Succession, and Statistical Inference , 1927 .

[16]  Yoav Freund,et al.  Boosting a weak learning algorithm by majority , 1990, COLT '90.

[17]  Xiaojin Zhu,et al.  Introduction to Semi-Supervised Learning , 2009, Synthesis Lectures on Artificial Intelligence and Machine Learning.

[18]  Alexander Zien,et al.  Semi-Supervised Classification by Low Density Separation , 2005, AISTATS.

[19]  Yoav Freund,et al.  Boosting the margin: A new explanation for the effectiveness of voting methods , 1997, ICML.

[20]  Thorsten Joachims,et al.  Transductive Inference for Text Classification using Support Vector Machines , 1999, ICML.

[21]  Horst Bischof,et al.  Semi-supervised On-Line Boosting for Robust Tracking , 2008, ECCV.

[22]  V. Vapnik.  Estimation of Dependences Based on Empirical Data , 2006 .