Adapting Kernels by Variational Approach in SVM

This paper proposed a variational Bayesian approach for the SVM regression based on the likelihood model of an infinite mixture of Gaussians. To evaluate this approach the method was applied to synthetic datasets. We compared this new approximation approach with the standard SVM algorithm as well as other well established methods such as Gaussian Process.

[1]  Matthias Seeger,et al.  Relationships between Gaussian processes, Support Vector machines and Smoothing Splines , 2000 .

[2]  Michael I. Jordan,et al.  Variational probabilistic inference and the QMR-DT database , 1998 .

[3]  David J. C. MacKay,et al.  Bayesian Methods for Backpropagation Networks , 1996 .

[4]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[5]  Peter Sollich.  Approximate learning curves for Gaussian processes , 1999 .

[6]  Massimiliano Pontil,et al.  On the Noise Model of Support Vector Machines Regression , 2000, ALT.

[7]  Shun-ichi Amari,et al.  Network information criterion-determining the number of hidden units for an artificial neural network model , 1994, IEEE Trans. Neural Networks.

[8]  S. Roberts,et al.  Variational Bayes for non-Gaussian autoregressive models , 2000, Neural Networks for Signal Processing X. Proceedings of the 2000 IEEE Signal Processing Society Workshop (Cat. No.00TH8501).

[9]  Tomaso Poggio,et al.  A Unified Framework for Regularization Networks and Support Vector Machines , 1999 .

[10]  David Barber,et al.  Bayesian Classification With Gaussian Processes , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[12]  Carl E. Rasmussen,et al.  In Advances in Neural Information Processing Systems , 2011 .

[13]  Christopher K. I. Williams.  Regression with Gaussian processes , 1997 .

[14]  Ole Winther,et al.  Gaussian Processes for Classification: Mean-Field Algorithms , 2000, Neural Computation.

[15]  L. Galway.  Spline Models for Observational Data , 1991 .

[16]  Hans Henrik Thodberg,et al.  A review of Bayesian neural networks with an application to near infrared spectroscopy , 1996, IEEE Trans. Neural Networks.

[17]  Bernhard Schölkopf,et al.  Learning with kernels , 2001 .

[18]  Michael I. Jordan.  Learning in Graphical Models , 1999, NATO ASI Series.

[19]  Peter Sollich.  Probabilistic interpretations and Bayesian methods for support vector machines , 1999 .

[20]  Christopher K. I. Williams.  Prediction with Gaussian Processes: From Linear Regression to Linear Prediction and Beyond , 1999, Learning in Graphical Models.

[21]  H. Akaike.  A new look at the statistical model identification , 1974 .

[22]  Neil D. Lawrence,et al.  Variational Bayesian Independent Component Analysis , 1999 .

[23]  Jerome H. Friedman.  Multivariate adaptive regression splines (with discussion) , 1991 .

[24]  Michael I. Jordan,et al.  Variational Probabilistic Inference and the QMR-DT Network , 2011, J. Artif. Intell. Res..

[25]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[26]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[27]  Christopher K. I. Williams.  Computing with Infinite Networks , 1996, NIPS.

[28]  Junbin Gao,et al.  A Probabilistic Framework for SVM Regression and Error Bar Estimation , 2002, Machine Learning.

[29]  David Barber,et al.  Gaussian Processes for Bayesian Classification via Hybrid Monte Carlo , 1996, NIPS.

[30]  Radford M. Neal.  Monte Carlo Implementation of Gaussian Process Models for Bayesian Regression and Classification , 1997, physics/9701026.

[31]  Geoffrey E. Hinton,et al.  Bayesian Learning for Neural Networks , 1995 .

[32]  Tomaso A. Poggio,et al.  Regularization Theory and Neural Networks Architectures , 1995, Neural Computation.

[33]  J. Friedman.  Multivariate adaptive regression splines , 1990 .

[34]  Charles M. Bishop.  Variational principal components , 1999 .

[35]  F. Girosi.  Models of Noise and Robust Estimates , 1991 .

[36]  Geoffrey E. Hinton,et al.  Evaluation of Gaussian processes and other methods for non-linear regression , 1997 .