Manifold regularization based semisupervised semiparametric regression

Semiparametric regression is an attractive solution for many practical problems. A nonlinear parametric model set, which is frequently encountered in practice, brings difficulties in complexity evaluation and control. On the other hand, it is quite common in practice that a mass of unlabeled samples is available, this fact suggests the possibility of applying a semisupervised regression method. Motivated by these facts, this paper proposes the manifold regularization based semisupervised semiparametric regression (MRBS^2R) method, which is characterized by introducing the manifold regularization (MR) technique in determining the parametric model. Generalization performance analysis shows that the generalization performance of the regression will be remarkably improved by introducing MR. Numerical experiments are performed to validate the proposed method.

[1]  Alexander J. Smola,et al.  Learning with Kernels: support vector machines, regularization, optimization, and beyond , 2001, Adaptive computation and machine learning series.

[2]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[3]  Edward Y. Chang,et al.  Semiparametric Regression Using Student $t$ Processes , 2007, IEEE Transactions on Neural Networks.

[4]  Ya-Xiang Yuan,et al.  Optimization Theory and Methods: Nonlinear Programming , 2010 .

[5]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[6]  John Shawe-Taylor,et al.  Covering numbers for support vector machines , 2002, IEEE Trans. Inf. Theory.

[7]  Meng Wang,et al.  Semi-Supervised Kernel Regression , 2006, Sixth International Conference on Data Mining (ICDM'06).

[8]  Zoubin Ghahramani,et al.  Learning from labeled and unlabeled data with label propagation , 2002 .

[9]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[10]  Bernhard Schölkopf,et al.  Semiparametric Support Vector and Linear Programming Machines , 1998, NIPS.

[11]  John D. Lafferty,et al.  Diffusion Kernels on Graphs and Other Discrete Input Spaces , 2002, ICML.

[12]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[13]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[14]  Xiaojin Zhu,et al.  --1 CONTENTS , 2006 .

[15]  Adam Krzyzak,et al.  A Distribution-Free Theory of Nonparametric Regression , 2002, Springer series in statistics.

[16]  Vikas Sindhwani,et al.  On Manifold Regularization , 2005, AISTATS.

[17]  B. Carl,et al.  Entropy, Compactness and the Approximation of Operators , 1990 .

[18]  Mikhail Belkin,et al.  Towards a theoretical foundation for Laplacian-based manifold methods , 2005, J. Comput. Syst. Sci..

[19]  Mikhail Belkin,et al.  Regularization and Semi-supervised Learning on Large Graphs , 2004, COLT.

[20]  Edward B. Fowlkes,et al.  Risk analysis of the space shuttle: Pre-Challenger prediction of failure , 1989 .

[21]  M. L. Berenson,et al.  Applied Statistics: A First Course. , 1989 .

[22]  Bernhard Schölkopf,et al.  Generalization Performance of Regularization Networks and Support Vector Machines via Entropy Numbers of Compact Operators , 1998 .

[23]  Bernhard Schölkopf,et al.  Learning from Labeled and Unlabeled Data Using Random Walks , 2004, DAGM-Symposium.

[24]  E. Kreyszig.  Introductory Functional Analysis With Applications , 1978 .

[25]  Nello Cristianini,et al.  Generalization Performance of Classifiers in Terms of Observed Covering Numbers , 1999, EuroCOLT.

[26]  C. F. Jeff Wu,et al.  Experiments: Planning, Analysis, and Parameter Design Optimization , 2000 .

[27]  I-Cheng Yeh,et al.  Modeling of strength of high-performance concrete using artificial neural networks , 1998 .

[28]  Alexander Zien,et al.  Semi-Supervised Learning , 2006 .

[29]  Lasse Holmström,et al.  A semiparametric density estimation approach to pattern classification , 2004, Pattern Recognit..