On Discriminative Semi-Supervised Classification

The recent years have witnessed a surge of interests in semi-supervised learning methods. A common strategy for these algorithms is to require that the predicted data labels should be sufficiently smooth with respect to the intrinsic data manifold. In this paper, we argue that rather than penalizing the label smoothness, we can directly punish the discriminality of the classification function to achieve a more powerful predictor, and we derive two specific algorithms: Semi-Supervised Discriminative Regularization (SSDR) and Semi-parametric Discriminative Semi-supervised Classification (SDSC). Finally many experimental results are presented to show the effectiveness of our method.

[1]  Alexander Zien,et al.  Semi-Supervised Learning , 2006 .

[2]  Mikhail Belkin,et al.  Beyond the point cloud: from transductive to semi-supervised learning , 2005, ICML.

[3]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[4]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[5]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[6]  Avrim Blum,et al.  Learning from Labeled and Unlabeled Data using Graph Mincuts , 2001, ICML.

[7]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[8]  David J. Miller,et al.  A Mixture of Experts Classifier with Learning Based on Both Labelled and Unlabelled Data , 1996, NIPS.

[9]  Eytan Domany,et al.  Semi-Supervised Learning -- A Statistical Physics Approach , 2006, ArXiv.

[10]  Mikhail Belkin,et al.  Regularization and Semi-supervised Learning on Large Graphs , 2004, COLT.

[11]  J. Friedman.  Regularized Discriminant Analysis , 1989 .

[12]  Tommi S. Jaakkola,et al.  Partially labeled classification with Markov random walks , 2001, NIPS.

[13]  David A. Landgrebe,et al.  The effect of unlabeled samples in reducing the small sample size problem and mitigating the Hughes phenomenon , 1994, IEEE Trans. Geosci. Remote. Sens..

[14]  Bernhard Schölkopf,et al.  Support vector channel selection in BCI , 2004, IEEE Transactions on Biomedical Engineering.

[15]  Sebastian Thrun,et al.  Text Classification from Labeled and Unlabeled Documents using EM , 2000, Machine Learning.

[16]  Thorsten Joachims,et al.  Transductive Inference for Text Classification using Support Vector Machines , 1999, ICML.

[17]  Mikhail Belkin,et al.  Manifold Regularization : A Geometric Framework for Learning from Examples , 2004 .

[18]  Ole Winther,et al.  Semi-Supervised Mean Fields , 2007, AISTATS.

[19]  Gene H. Golub,et al.  Matrix computations , 1983 .

[20]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[21]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[22]  Bernhard Schölkopf,et al.  Transductive Classification via Local Learning Regularization , 2007, AISTATS.

[23]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[24]  Fan Chung,et al.  Spectral Graph Theory , 1996 .

[25]  Thomas Hofmann,et al.  Semi-supervised Learning on Directed Graphs , 2004, NIPS.