Leave-One-Out Cross-Validation Based Model Selection for Manifold Regularization

Classified labels are expensive by virtue of the utilization of field knowledge while the unlabeled data contains significant information, which can not be explored by supervised learning The Manifold Regularization (MR) based semi-supervised learning (SSL) could explores information from both labeled and unlabeled data Moreover, the model selection of MR seriously affects its predictive performance due to the inherent additional geometry regularizer of SSL In this paper, a leave-one-out cross-validation based PRESS criterion is first presented for model selection of MR to choose appropriate regularization coefficients and kernel parameters The Manifold regularization and model selection algorithm are employed to a real-life benchmark dataset The proposed approach, leveraged by effectively exploiting the embedded intrinsic geometric manifolds, outperforms the original MR and supervised learning approaches.

[1]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[2]  Liefeng Bo,et al.  Feature Scaling for Kernel Fisher Discriminant Analysis Using Leave-One-Out Cross Validation , 2006 .

[3]  Alexander Zien,et al.  Semi-Supervised Learning , 2006 .

[4]  Gavin C. Cawley,et al.  Leave-One-Out Cross-Validation Based Model Selection Criteria for Weighted LS-SVMs , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[5]  Andrew Blake,et al.  Sparse and Semi-supervised Visual Mapping with the S^3GP , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[6]  Jinwu Xu,et al.  Multiple manifolds analysis and its application to fault diagnosis , 2009 .

[7]  Jeffrey C. Lagarias,et al.  Convergence Properties of the Nelder-Mead Simplex Method in Low Dimensions , 1998, SIAM J. Optim..

[8]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[9]  David M. Allen,et al.  The Relationship Between Variable Selection and Data Agumentation and a Method for Prediction , 1974 .

[10]  Vikas Sindhwani,et al.  On semi-supervised kernel methods , 2007 .

[11]  Qiang Yang,et al.  A Manifold Regularization Approach to Calibration Reduction for Sensor-Network Based Tracking , 2006, AAAI.

[12]  Alexander Zien,et al.  Semi-Supervised Classification by Low Density Separation , 2005, AISTATS.

[13]  Xiaojin Zhu,et al.  --1 CONTENTS , 2006 .

[14]  Tao Yu,et al.  Reliable multi-objective optimization of high-speed WEDM process based on Gaussian process regression , 2008 .

[15]  Li Wei,et al.  Semi-supervised time series classification , 2006, KDD '06.

[16]  Mikhail Belkin,et al.  Beyond the point cloud: from transductive to semi-supervised learning , 2005, ICML.