Ordinal Regression with Sparse Bayesian

In this paper, a probabilistic framework for ordinal prediction is proposed, which can be used in modeling ordinal regression. A sparse Bayesian treatment for ordinal regression is given by us, in which an automatic relevance determination prior over weights is used. The inference techniques based on Laplace approximation is adopted for model selection. By this approach accurate prediction models can be derived, which typically utilize dramatically fewer basis functions than the comparable supported vector based and Gaussian process based approaches while offering a number of additional advantages. Experimental results on the real-world data set show that the generalization performance competitive with support vector-based method and Gaussian process-based method.

[1]  Andrew Blake,et al.  Sparse Bayesian learning for efficient visual tracking , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Ling Li,et al.  Ordinal Regression by Extended Binary Classification , 2006, NIPS.

[3]  Michael E. Tipping Sparse Bayesian Learning and the Relevance Vector Machine , 2001, J. Mach. Learn. Res..

[4]  Geoffrey E. Hinton,et al.  Bayesian Learning for Neural Networks , 1995 .

[5]  Amnon Shashua,et al.  Ranking with Large Margin Principle: Two Approaches , 2002, NIPS.

[6]  Cheng-Yuan Liou,et al.  Dynamic Positron Emission Tomography Data-Driven Analysis Using Sparse Bayesian Learning , 2008, IEEE Transactions on Medical Imaging.

[7]  Gavin C. Cawley,et al.  Sparse bayesian kernel survival analysis for modeling the growth domain of microbial pathogens , 2006, IEEE Transactions on Neural Networks.

[8]  Robert M. Nishikawa,et al.  A relevance vector machine technique for the automatic detection of clustered microcalcifications (Honorable Mention Poster Award) , 2005, SPIE Medical Imaging.

[9]  Chris Buckley,et al.  OHSUMED: an interactive retrieval evaluation and new large test collection for research , 1994, SIGIR '94.

[10]  Yi Li,et al.  Bayesian automatic relevance determination algorithms for classifying gene expression data. , 2002, Bioinformatics.

[11]  Eibe Frank,et al.  A Simple Approach to Ordinal Classification , 2001, ECML.

[12]  Wei Chu,et al.  Minimum Enclosing Spheres Formulations for Support Vector Ordinal Regression , 2006, Sixth International Conference on Data Mining (ICDM'06).

[13]  Tao Qin,et al.  LETOR: Benchmark Dataset for Research on Learning to Rank for Information Retrieval , 2007 .

[14]  Gerhard Widmer,et al.  Prediction of Ordinal Classes Using Regression Trees , 2001, Fundam. Informaticae.

[15]  Edward F. Harrington,et al.  Online Ranking/Collaborative Filtering Using the Perceptron Algorithm , 2003, ICML.

[16]  B. Triggs,et al.  3D human pose from silhouettes by relevance vector regression , 2004, CVPR 2004.

[17]  David J. C. MacKay,et al.  A Practical Bayesian Framework for Backpropagation Networks , 1992, Neural Computation.

[18]  Wei Chu,et al.  Gaussian Processes for Ordinal Regression , 2005, J. Mach. Learn. Res..

[19]  Wei Chu,et al.  New approaches to support vector ordinal regression , 2005, ICML.

[20]  Jaana Kekäläinen,et al.  Binary and graded relevance in IR evaluations--Comparison of the effects on ranking of IR systems , 2005, Inf. Process. Manag..

[21]  Koby Crammer,et al.  Pranking with Ranking , 2001, NIPS.