Bidirectional Semi-supervised Learning with Graphs

We present a machine learning task, which we call bidirectional semi-supervised learning, where label-only samples are given as well as labeled and unlabeled samples. A label-only sample contains the label information of the sample but not the feature information. Then, we propose a simple and effective graph-based method for bidirectional semi-supervised learning in multi-label classification. The proposed method assumes that correlated classes are likely to have the same labels among the similar samples. First, we construct a graph that represents similarities between samples using labeled and unlabeled samples in the same way with graph-based semi-supervised methods. Second, we construct another graph using labeled and label-only samples by connecting classes that are likely to co-occur, which represents correlations between classes. Then, we estimate labels of unlabeled samples by propagating labels over these two graphs. We can find a closed-form global solution for the label propagation by using matrix algebra. We demonstrate the effectiveness of the proposed method over supervised and semi-supervised learning methods with experiments using synthetic and multi-label text data sets.

[1]  Hirokazu Kameoka,et al.  SemiCCA: Efficient Semi-supervised Learning of Canonical Correlations , 2010, International Conference on Pattern Recognition.

[2]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[3]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[4]  Yasuhiro Fujiwara,et al.  Recommendations Over Domain Specific User Graphs , 2010, ECAI.

[5]  Gerhard Weikum,et al.  Graffiti: graph-based classification in heterogeneous networks , 2011, World Wide Web.

[6]  Xian-Sheng Hua,et al.  Transductive multi-label learning for video concept detection , 2008, MIR '08.

[7]  John C. Platt,et al.  Translingual Document Representations from Discriminative Projections , 2010, EMNLP.

[8]  Xue-wen Chen,et al.  Mr.KNN: soft relevance for multi-label classification , 2010, CIKM.

[9]  Alexander Zien,et al.  Semi-Supervised Learning , 2006 .

[10]  Tao Mei,et al.  Local-driven semi-supervised learning with multi-label , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[11]  Daniel Marcu,et al.  Domain Adaptation for Statistical Classifiers , 2006, J. Artif. Intell. Res..

[12]  Jiebo Luo,et al.  Annotating photo collections by label propagation according to multiple similarity cues , 2008, ACM Multimedia.

[13]  Tao Mei,et al.  Graph-based semi-supervised learning with multiple labels , 2009, J. Vis. Commun. Image Represent..

[14]  O. Mangasarian,et al.  Semi-Supervised Support Vector Machines for Unlabeled Data Classification , 2001 .

[15]  Yiming Yang,et al.  Multilabel classification with meta-level features , 2010, SIGIR.

[16]  Ayhan Demiriz,et al.  Semi-Supervised Support Vector Machines , 1998, NIPS.

[17]  Wei Liu,et al.  Robust multi-class transductive learning with graphs , 2009, CVPR.

[18]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[19]  Jinxi Xu,et al.  Evaluating a probabilistic model for cross-lingual information retrieval , 2001, SIGIR '01.

[20]  Masashi Sugiyama,et al.  Robust Label Propagation on Multiple Networks , 2009, IEEE Transactions on Neural Networks.

[21]  Naonori Ueda,et al.  Semisupervised Learning for a Hybrid Generative/Discriminative Classifier based on the Maximum Entropy Principle , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Susan T. Dumais,et al.  Automatic cross-linguistic information retrieval using latent semantic indexing , 2007 .

[23]  Zoubin Ghahramani,et al.  Learning from labeled and unlabeled data with label propagation , 2002 .

[24]  Nando de Freitas,et al.  Fast Computational Methods for Visually Guided Robots , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[25]  Xiaojin Zhu,et al.  Introduction to Semi-Supervised Learning , 2009, Synthesis Lectures on Artificial Intelligence and Machine Learning.

[26]  Nicolas Le Roux,et al.  Label Propagation and Quadratic Criterion , 2006, Semi-Supervised Learning.

[27]  Fei-Fei Li,et al.  Connecting modalities: Semi-supervised segmentation and annotation of images using unaligned text corpora , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[28]  Tatsushi Matsubayashi,et al.  Hierarchical Individual time algorithm for large scale graph-drawing , 2007 .

[29]  Jeff A. Bilmes,et al.  Soft-Supervised Learning for Text Classification , 2008, EMNLP.

[30]  Sebastian Thrun,et al.  Text Classification from Labeled and Unlabeled Documents using EM , 2000, Machine Learning.

[31]  Michael I. Jordan,et al.  Modeling annotated data , 2003, SIGIR.

[32]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[33]  M. Griebel,et al.  Semi-supervised learning with sparse grids , 2005, ICML 2005.

[34]  Naonori Ueda,et al.  Improving Classifier Performance Using Data with Different Taxonomies , 2011, IEEE Transactions on Knowledge and Data Engineering.

[35]  Yi Liu,et al.  Semi-supervised Multi-label Learning by Constrained Non-negative Matrix Factorization , 2006, AAAI.

[36]  Mikhail Belkin,et al.  Regularization and Semi-supervised Learning on Large Graphs , 2004, COLT.

[37]  Tatsushi Matsubayashi,et al.  A Force-directed Graph Drawing based on the Hierarchical Individual Timestep Method , 2007 .

[38]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[39]  Andrew McCallum,et al.  Using Maximum Entropy for Text Classification , 1999 .