Fully Bayesian Analysis of Relevance Vector Machine Classification With Probit Link Function for Imbalanced Data Problem

The original RVM classification model uses the logistic link function to build the likelihood function making the model hard to be conducted since the posterior of the weight parameter has no closed-form solution. This article proposes the probit link function approach instead of the logistic one for the likelihood function in the RVM classification model, namely PRVM (RVM with the probit link function). We show that the posterior of the weight parameter in PRVM follows the Multivariate Normal distribution and achieves a closed-form solution. A latent variable is needed in our algorithms to simplify the Bayesian computation greatly, and its conditional posterior follows a truncated Normal distribution. Compared with the original RVM classification model, our proposed one is a Fully Bayesian approach, and it has a more efficient computation process. For the prior structure, we first consider the Normal-Gamma independent prior to propose a Generic Bayesian PRVM algorithm. Furthermore, the Fully Bayesian PRVM algorithm with a hierarchical hyperprior structure is proposed, which improves the classification performance, especially in the imbalanced data problem.

[1]  Jesús Alcalá-Fdez,et al.  KEEL Data-Mining Software Tool: Data Set Repository, Integration of Algorithms and Experimental Analysis Framework , 2011, J. Multiple Valued Log. Soft Comput..

[2]  Giles M. Foody,et al.  Evaluation of SVM, RVM and SMLR for Accurate Image Classification With Limited Ground Data , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[3]  S. Chib,et al.  Bayesian residual analysis for binary response regression models , 1995 .

[4]  Michael E. Tipping.  The Relevance Vector Machine , 1999, NIPS.

[5]  Xu Xiang-min,et al.  Classification Performance Comparison between RVM and SVM , 2007, 2007 International Workshop on Anti-Counterfeiting, Security and Identification (ASID).

[6]  Begüm Demir,et al.  Hyperspectral Image Classification Using Relevance Vector Machines , 2007, IEEE Geoscience and Remote Sensing Letters.

[7]  Di Cai,et al.  Multivariate Seawater Quality Prediction Based on PCA-RVM Supported by Edge Computing Towards Smart Ocean , 2020, IEEE Access.

[8]  By W. R. GILKSt,et al.  Adaptive Rejection Sampling for Gibbs Sampling , 2010 .

[9]  Peng Chen,et al.  Driving Fatigue Classification Based on Fusion Entropy Analysis Combining EOG and EEG , 2019, IEEE Access.

[10]  James Clary,et al.  Stochastic theory of minimal realization , 1976, 1976 IEEE Conference on Decision and Control including the 15th Symposium on Adaptive Processes.

[11]  N. Nikolaev,et al.  Sequential relevance vector machine learning from time series , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[12]  B. Schölkopf,et al.  Advances in kernel methods: support vector learning , 1999 .

[13]  Dongchu Sun,et al.  Fully Bayesian analysis of the relevance vector machine with an extended hierarchical prior structure , 2011 .

[14]  G. Casella,et al.  Generalized Accept-Reject sampling schemes , 2004 .

[15]  W. Gilks,et al.  Adaptive Rejection Metropolis Sampling Within Gibbs Sampling , 1995 .

[16]  Uwe Weidner,et al.  Classification in High-Dimensional Feature Spaces—Assessment Using SVM, IVM and RVM With Focus on Simulated EnMAP Data , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[17]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevance Vector Machine , 2001 .

[18]  Lin Tao,et al.  Incipient Fault Diagnosis Method for IGBT Drive Circuit Based on Improved SAE , 2019, IEEE Access.

[19]  Bernardete Ribeiro,et al.  Scaling Text Classification with Relevance Vector Machines , 2006, 2006 IEEE International Conference on Systems, Man and Cybernetics.

[20]  Stephen C. Graves,et al.  Scheduling Policies for Automatic Warehousing Systems: Simulation Results , 1978 .

[21]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[22]  Y. Teh,et al.  Indian Buffet Processes with Power-law Behavior , 2009, NIPS.

[23]  Albert J. Kinderman,et al.  Computer Generation of Random Variables Using the Ratio of Uniform Deviates , 1977, TOMS.

[24]  Michael E. Tipping,et al.  Fast Marginal Likelihood Maximisation for Sparse Bayesian Models , 2003 .

[25]  Wenyang Wang,et al.  Fully Bayesian Analysis of the Relevance Vector Machine Classification for Imbalanced Data , 2020, CAAI Transactions on Intelligence Technology.

[26]  Michael H. Goldbaum,et al.  Convex Representations Using Deep Archetypal Analysis for Predicting Glaucoma , 2020, IEEE Journal of Translational Engineering in Health and Medicine.

[27]  Brian D. Ripley,et al.  Regression techniques for the detection of analytical bias , 1987 .