Numerical variable reconstruction from ordinal categories based on probability distributions

Ordinal classification problems are an active research area in the machine learning community. Many previous works adapted state-of-art nominal classifiers to improve ordinal classification so that the method can take advantage of the ordinal structure of the dataset. However, these method improvements often rely upon a complex mathematical basis and they usually are attached to the training algorithm and model. This paper presents a novel method for generally adapting classification and regression models, such as artificial neural networks or support vector machines. The ordinal classification problem is reformulated as a regression problem by the reconstruction of a numerical variable which represents the different ordered class labels. Despite the simplicity and generality of the method, results are competitive in comparison with very specific methods for ordinal regression.

[1]  Gerhard Widmer,et al.  Prediction of Ordinal Classes Using Regression Trees , 2001, Fundam. Informaticae.

[2]  Vladimir Vapnik,et al.  An overview of statistical learning theory , 1999, IEEE Trans. Neural Networks.

[3]  Geoffrey E. Hinton,et al.  Bayesian Learning for Neural Networks , 1995 .

[4]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[5]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[6]  Koby Crammer,et al.  Pranking with Ranking , 2001, NIPS.

[7]  Ling Li,et al.  Ordinal Regression by Extended Binary Classification , 2006, NIPS.

[8]  P. McCullagh.  Regression Models for Ordinal Data , 1980 .

[9]  Willem Waegeman,et al.  An ensemble of Weighted Support Vector Machines for Ordinal Regression , 2007 .

[10]  David J. C. MacKay,et al.  Bayesian Methods for Backpropagation Networks , 1996 .

[11]  Ramani Duraiswami,et al.  A Fast Algorithm for Learning a Ranking Function from Large-Scale Data Sets , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  L. Devroye.  Non-Uniform Random Variate Generation , 1986 .

[13]  Wei Chu,et al.  Gaussian Processes for Ordinal Regression , 2005, J. Mach. Learn. Res..

[14]  Wei Chu,et al.  New approaches to support vector ordinal regression , 2005, ICML.

[15]  Bernard De Baets,et al.  Learning partial ordinal class memberships with kernel-based proportional odds models , 2012, Comput. Stat. Data Anal..

[16]  M. Friedman.  A Comparison of Alternative Tests of Significance for the Problem of $m$ Rankings , 1940 .

[17]  Wei Chu,et al.  Support Vector Ordinal Regression , 2007, Neural Computation.

[18]  Eibe Frank,et al.  A Simple Approach to Ordinal Classification , 2001, ECML.

[19]  Thore Graepel,et al.  Large Margin Rank Boundaries for Ordinal Regression , 2000 .