A Probabilistic Framework for SVM Regression and Error Bar Estimation

In this paper, we elaborate on the well-known relationship between Gaussian Processes (GP) and Support Vector Machines (SVM) under some convex assumptions for the loss functions. This paper concentrates on the derivation of the evidence and error bar approximation for regression problems. An error bar formula is derived based on the ∈-insensitive loss function.

[1]  Vladimir Vapnik,et al.  The Nature of Statistical Learning , 1995 .

[2]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[3]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[4]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[5]  Peter Sollich.  Probabilistic interpretations and Bayesian methods for support vector machines , 1999 .

[6]  G. Wahba.  Support Vector Machines, Reproducing Kernel Hilbert Spaces and the Randomized GACV 1 , 1998 .

[7]  Christopher K. I. Williams.  Computing with Infinite Networks , 1996, NIPS.

[8]  Tomaso Poggio,et al.  A Unified Framework for Regularization Networks and Support Vector Machines , 1999 .

[10]  S. Amari,et al.  Network Information Criterion | Determining the Number of Hidden Units for an Articial Neural Network Model Network Information Criterion | Determining the Number of Hidden Units for an Articial Neural Network Model , 2007 .

[11]  Federico Girosi,et al.  An Equivalence Between Sparse Approximation and Support Vector Machines , 1998, Neural Computation.

[12]  Geoffrey E. Hinton,et al.  Bayesian Learning for Neural Networks , 1995 .

[13]  Peter Sollich.  Approximate learning curves for Gaussian processes , 1999 .

[14]  Grace Wahba,et al.  Spline Models for Observational Data , 1990 .

[15]  James T. Kwok.  Moderating the outputs of support vector machine classifiers , 1999, IEEE Trans. Neural Networks.

[16]  Martin Brown,et al.  Network Performance Assessment for Neurofuzzy Data Modelling , 1997, IDA.

[17]  Christopher K. I. Williams.  Prediction with Gaussian Processes: From Linear Regression to Linear Prediction and Beyond , 1999, Learning in Graphical Models.

[18]  H. Akaike.  A new look at the statistical model identification , 1974 .

[19]  David J. C. MacKay,et al.  Bayesian Interpolation , 1992, Neural Computation.

[20]  Tomaso A. Poggio,et al.  A Sparse Representation for Function Approximation , 1998, Neural Computation.

[21]  Peter Sollich,et al.  Probabilistic Methods for Support Vector Machines , 1999, NIPS.

[22]  Bernhard Schölkopf,et al.  Learning with kernels , 2001 .