Semi-Supervised Classification Using Linear Neighborhood Propagation

In this paper, we address the general problem of learning from both labeled and unlabeled data. Based on the reasonable assumption that the label of each data can be linearly reconstructed from its neighbors’ labels, we develop a novel approach, called Linear Neighborhood Propagation (LNP), to learn the linear construction weights and predict the labels of the unlabeled data from the labeled data. The main procedure is composed of two steps: (1) it computes the local neighborhood geometry according to the image information by solving a constrained least squares problem, (2) the obtained geometry will be used to predict the labels of unlabeled data by solving a linearly constrained quadratic optimization problem. LNP has at least three characteristics. Firstly, it is much practical in that only one free parameter, the size of neighborhood, requires user-tuning. Secondly it can be easily extended to out-of-sample examples. Finally the labels achieved from LNP can be sufficiently smooth with respect to the intrinsic structure collectively revealed by the labeled and unlabeled points. Many promising experimental results, such as object recognition, data ranking and interactive image segmentation, demonstrate the effectiveness and efficiency of the proposed approach.

[1]  Leo Grady,et al.  Multilabel random walker image segmentation using prior models , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[2]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[3]  Jian Sun,et al.  Lazy snapping , 2004, SIGGRAPH 2004.

[4]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[5]  Nicolas Le Roux,et al.  Efficient Non-Parametric Function Induction in Semi-Supervised Learning , 2004, AISTATS.

[6]  Bernhard Schölkopf,et al.  Ranking on Data Manifolds , 2003, NIPS.

[7]  Mikhail Belkin,et al.  Tikhonov regularization and semi-supervised learning on large graphs , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[8]  Marie-Pierre Jolly,et al.  Interactive Graph Cuts for Optimal Boundary and Region Segmentation of Objects in N-D Images , 2001, ICCV.

[9]  Changshui Zhang,et al.  Spectral feature analysis , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[10]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[11]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Lawrence K. Saul,et al.  Think Globally, Fit Locally: Unsupervised Learning of Low Dimensional Manifold , 2003, J. Mach. Learn. Res..

[13]  Mikhail Belkin,et al.  Regularization and Semi-supervised Learning on Large Graphs , 2004, COLT.

[14]  Jean-Marc Constans,et al.  FUZZY SEGMENTATION OF CEREBRAL TUMOROUS TISSUES IN MR IMAGES VIA SUPPORT VECTOR MACHINE AND FUZZY CLUSTERING , 2005 .

[15]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[16]  Mikhail Belkin,et al.  Semi-Supervised Learning on Riemannian Manifolds , 2004, Machine Learning.

[17]  Andy Harter,et al.  Parameterisation of a stochastic model for human face identification , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[18]  Vikas Sindhwani,et al.  On Manifold Regularization , 2005, AISTATS.

[19]  Bernhard Schölkopf,et al.  Cluster Kernels for Semi-Supervised Learning , 2002, NIPS.

[20]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[21]  PaperNo.  Recognition of shapes by editing shock graphs , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[22]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[23]  Thorsten Joachims,et al.  Transductive Inference for Text Classification using Support Vector Machines , 1999, ICML.

[24]  Avrim Blum,et al.  Learning from Labeled and Unlabeled Data using Graph Mincuts , 2001, ICML.

[25]  Dani Lischinski,et al.  Colorization using optimization , 2004, ACM Trans. Graph..

[26]  Marie-Pierre Jolly,et al.  Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[27]  Dan Klein,et al.  Spectral Learning , 2003, IJCAI.

[28]  Ming-Hsuan Yang,et al.  Kernel Eigenfaces vs. Kernel Fisherfaces: Face recognition using kernel methods , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[29]  Tommi S. Jaakkola,et al.  Partially labeled classification with Markov random walks , 2001, NIPS.

[30]  Bernhard Schölkopf,et al.  Regularization on Discrete Spaces , 2005, DAGM-Symposium.

[31]  Philip N. Klein,et al.  Recognition of shapes by editing their shock graphs , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[33]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.