Semi-supervised learning based on high density region estimation

In this paper, we consider local regression problems on high density regions. We propose a semi-supervised local empirical risk minimization algorithm and bound its generalization error. The theoretical analysis shows that our method can utilize unlabeled data effectively and achieve fast learning rate.

[1]  A. Cuevas,et al.  A plug-in approach to support estimation , 1997 .

[2]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[3]  Zongben Xu,et al.  The generalization performance of ERM algorithm with strongly mixing observations , 2009, Machine Learning.

[4]  Ding-Xuan Zhou,et al.  The covering number in learning theory , 2002, J. Complex..

[5]  Philippe Rigollet,et al.  Generalization Error Bounds in Semi-supervised Classification Under the Cluster Assumption , 2006, J. Mach. Learn. Res..

[6]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[7]  Tong Zhang,et al.  On the Effectiveness of Laplacian Normalization for Graph Semi-supervised Learning , 2007, J. Mach. Learn. Res..

[8]  Masashi Sugiyama,et al.  Input-dependent estimation of generalization error under covariate shift , 2005 .

[9]  Radu Herbei,et al.  Classification with reject option , 2006 .

[10]  Ding-Xuan Zhou,et al.  Learning and approximation by Gaussians on Riemannian manifolds , 2009, Adv. Comput. Math..

[11]  Luoqing Li,et al.  Semisupervised Multicategory Classification With Imperfect Model , 2009, IEEE Transactions on Neural Networks.

[12]  Ding-Xuan Zhou,et al.  Capacity of reproducing kernel spaces in learning theory , 2003, IEEE Transactions on Information Theory.

[13]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[14]  Hans-Peter Kriegel,et al.  OPTICS: ordering points to identify the clustering structure , 1999, SIGMOD '99.

[15]  Felipe Cucker,et al.  Learning Theory: An Approximation Theory Viewpoint: Index , 2007 .

[16]  Junhui Wang,et al.  Large Margin Semi-supervised Learning , 2007, J. Mach. Learn. Res..

[17]  Motoaki Kawanabe,et al.  Trading Variance Reduction with Unbiasedness: The Regularized Subspace Information Criterion for Robust Model Selection in Kernel Regression , 2004, Neural Computation.

[18]  V. Koltchinskii,et al.  Concentration inequalities and asymptotic results for ratio type empirical processes , 2006, math/0606788.

[19]  Felipe Cucker,et al.  On the mathematical foundations of learning , 2001 .

[20]  Jiangtao Peng,et al.  Error bounds of multi-graph regularized semi-supervised classification , 2009, Inf. Sci..

[21]  Thorsten Joachims,et al.  Transductive Inference for Text Classification using Support Vector Machines , 1999, ICML.

[22]  Yuan Yan Tang,et al.  Analysis of Classification with a Reject Option , 2009, Int. J. Wavelets Multiresolution Inf. Process..

[23]  Ronald A. DeVore,et al.  Approximation Methods for Supervised Learning , 2006, Found. Comput. Math..

[24]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[25]  Luoqing Li,et al.  The performance bounds of learning machines based on exponentially strongly mixing sequences , 2007, Comput. Math. Appl..

[26]  Mikhail Belkin,et al.  Regularization and Semi-supervised Learning on Large Graphs , 2004, COLT.

[27]  OLIVlER 13OUSQUET,et al.  NEW APPROACHES TO STATISTICAL LEARNING THEORY , 2006 .

[28]  Xiaojin Zhu,et al.  --1 CONTENTS , 2006 .

[29]  Masashi Sugiyama,et al.  Optimal design of regularization term and regularization parameter by subspace information criterion , 2002, Neural Networks.

[30]  Tong Zhang,et al.  Graph-Based Semi-Supervised Learning and Spectral Kernel Design , 2008, IEEE Transactions on Information Theory.