Generalized relevance vector machine

This paper considers the generalized version of relevance vector machine (RVM), which is a sparse Bayesian kernel machine for classification and ordinary regression. Generalized RVM (GRVM) follows the work of generalized linear model (GLM), which is a natural generalization of ordinary linear regression model and shares a common approach to estimate the parameters. GRVM inherits the advantages of GLM, i.e., unified model structure, same training algorithm, and convenient task-specific model design. It also inherits the advantages of RVM, i.e., probabilistic output, extremely sparse solution, hyperparameter auto-estimation. Besides, GRVM extends RVM to a wider range of learning tasks beyond classification and ordinary regression by assuming that the conditional output belongs to exponential family distribution (EFD). Since EFD results in inference intractable problem in Bayesian analysis, in this paper, Laplace approximation is adopted to solve this problem, which is a common approach in Bayesian inference. Further, several task-specific models are designed based on GRVM including models for ordinary regression, count data regression, classification, ordinal regression, etc. Besides, the relationship between GRVM and traditional RVM models are discussed. Finally, experimental results show the efficiency of the proposed GRVM model.

[1]  Ye Zhang,et al.  Robust Hyperspectral Classification Using Relevance Vector Machine , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[2]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[3]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[4]  Begüm Demir,et al.  Hyperspectral Image Classification Using Relevance Vector Machines , 2007, IEEE Geoscience and Remote Sensing Letters.

[5]  Zhihua Zhang,et al.  Bayesian Generalized Kernel Mixed Models , 2011, J. Mach. Learn. Res..

[6]  Jong-Duk Son,et al.  Fault diagnosis of low speed bearing based on relevance vector machine and support vector machine , 2009, Expert Syst. Appl..

[7]  Zhihua Zhang,et al.  Bayesian Generalized Kernel Models , 2010, AISTATS.

[8]  Gavin C. Cawley,et al.  Generalised Kernel Machines , 2007, 2007 International Joint Conference on Neural Networks.

[9]  P. Gupta,et al.  Relevance Vector Machine for Optical Diagnosis of Cancer , 2022 .

[10]  Hamid Sheikhzadeh,et al.  Relevance Vector Machine for Survival Analysis , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[11]  Bo-Suk Yang,et al.  Application of relevance vector machine and logistic regression for machine degradation assessment , 2010 .

[12]  Enrico Zio,et al.  Fatigue crack growth estimation by relevance vector machine , 2012, Expert Syst. Appl..

[13]  Maja Pantic,et al.  Doubly Sparse Relevance Vector Machine for Continuous Facial Behavior Estimation , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Qinghua Zheng,et al.  Sparse Bayesian learning for ranking , 2009, 2009 IEEE International Conference on Granular Computing.

[15]  Antoni B. Chan,et al.  Generalized Gaussian process models , 2011, CVPR 2011.

[16]  Roni Khardon,et al.  Sparse variational inference for generalized Gaussian process models , 2015, ICML 2015.

[17]  Michael E. Tipping,et al.  Fast Marginal Likelihood Maximisation for Sparse Bayesian Models , 2003 .

[18]  Qinghua Zheng,et al.  Ordinal Regression with Sparse Bayesian , 2009, ICIC.

[19]  P. McCullagh,et al.  Generalized Linear Models , 1984 .

[20]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevance Vector Machine , 2001 .

[21]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[22]  Qingsong Xu,et al.  Rate-Dependent Hysteresis Modeling and Control of a Piezostage Using Online Support Vector Machine and Relevance Vector Machine , 2012, IEEE Transactions on Industrial Electronics.