Robust relevance vector machine with noise variance coefficient

Classical relevance vector machine is sensitive to outliers during training and has weak robustness. In order to solve this problem, a novel robust relevance vector machine is presented in this paper. The key idea of the proposed method is to introduce individual noise variance coefficient for each training sample. In the process of model training, the noise variance coefficients of outliers gradually decrease so as to automatically detect and eliminate outliers. In addition, the iterative formulae for the optimization of noise variance coefficients and hyperparameters are derived according to the Bayesian evidence framework. Simulation results on sinc function and some benchmark data sets demonstrate that the proposed robust relevance vector machine can resist the impact of outliers effectively and obtain better robustness in comparison with other methods.

[1]  Michael E. Tipping.  The Relevance Vector Machine , 1999, NIPS.

[2]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[3]  Youxian Sun,et al.  Soft Sensor Based on Relevance Vector Machines for Microbiological Fermentation , 2008 .

[4]  Gábor Horváth,et al.  A Sparse Robust Model for a Linz-Donawitz Steel Converter , 2007, 2007 IEEE Instrumentation & Measurement Technology Conference IMTC 2007.

[5]  Muhsin Tunay Gençoglu,et al.  Prediction of flashover voltage of insulators using least squares support vector machines , 2009, Expert Syst. Appl..

[6]  Shuqing Wang,et al.  Support vector machine based predictive functional control design for output temperature of coking furnace , 2008 .

[7]  Tao Yu,et al.  Integrating relevance vector machines and genetic algorithms for optimization of seed-separating process , 2007, Eng. Appl. Artif. Intell..

[8]  R. Govindaraju,et al.  On selection of kernel parametes in relevance vector machines for hydrologic applications , 2007 .

[9]  Robert M. Nishikawa,et al.  Relevance vector machine for automatic detection of clustered microcalcifications , 2005, IEEE Transactions on Medical Imaging.

[10]  Biao Yang,et al.  Robust Relevance Vector Regression With Trimmed Likelihood Function , 2007, IEEE Signal Processing Letters.

[11]  David J. C. MacKay,et al.  Bayesian Interpolation , 1992, Neural Computation.

[12]  N. Nikolaev,et al.  Sequential relevance vector machine learning from time series , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[13]  David J. C. MacKay,et al.  The Evidence Framework Applied to Classification Networks , 1992, Neural Computation.

[14]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevan e Ve tor Ma hine , 2001 .

[15]  Zhifeng Li,et al.  Bayesian face recognition using support vector machine and face clustering , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[16]  Tao Yu,et al.  Adaptive spherical Gaussian kernel in sparse Bayesian learning framework for nonlinear regression , 2009, Expert Syst. Appl..

[17]  Michael E. Tipping,et al.  A Variational Approach to Robust Regression , 2001, ICANN.

[18]  Gunnar Rätsch,et al.  An introduction to kernel-based learning algorithms , 2001, IEEE Trans. Neural Networks.

[19]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory, Second Edition , 2000, Statistics for Engineering and Information Science.

[20]  Neil D. Lawrence,et al.  Variational inference for Student-t models: Robust Bayesian interpolation and generalised component analysis , 2005, Neurocomputing.

[21]  Stefan Schaal,et al.  Automatic Outlier Detection: A Bayesian Approach , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.