Statistical models and learning algorithms for ordinal regression problems

In this study, we propose a learning algorithm for ordinal regression problems. In most existing learning algorithms, the threshold or location model is assumed to be the statistical model. For estimation of conditional probability of labels for a given covariate vector, we extended the location model to apply ordinal regressions. We present this learning algorithm using the squared-loss function with the location-scale models for estimating conditional probability. We prove that the estimated conditional probability satisfies the monotonicity of the distribution function. Furthermore, we have conducted numerical experiments to compare these proposed methods with existing approaches. We found that, in its ability to predict labels, our method may not have an advantage over existing approaches. However, for estimating conditional probabilities, it does outperform the learning algorithm using location models.

[1]  Koby Crammer,et al.  Pranking with Ranking , 2001, NIPS.

[2]  Wei Chu,et al.  Gaussian Processes for Ordinal Regression , 2005, J. Mach. Learn. Res..

[3]  P. Tseng Convergence of a Block Coordinate Descent Method for Nondifferentiable Minimization , 2001 .

[4]  Jaime S. Cardoso,et al.  The unimodal model for the classification of ordinal data , 2008, Neural Networks.

[5]  Wei Chu,et al.  New approaches to support vector ordinal regression , 2005, ICML.

[6]  I. Gertsbakh,et al.  ESTIMATION OF A STANDARD MEASURING ERROR BY REPEATED MEASUREMENTS AND SORTINGS , 1985 .

[7]  Ling Li,et al.  Ordinal Regression by Extended Binary Classification , 2006, NIPS.

[8]  Kurt Hornik,et al.  kernlab - An S4 Package for Kernel Methods in R , 2004 .

[9]  A. Shashua,et al.  Taxonomy of Large Margin Principle Algorithms for Ordinal Regression Problems , 2002 .

[10]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[11]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[12]  Alexander J. Smola,et al.  Learning with Kernels: support vector machines, regularization, optimization, and beyond , 2001, Adaptive computation and machine learning series.

[13]  David Ruppert,et al.  Robust Estimation in Heteroscedastic Linear Models. , 1982 .

[14]  Songnian Chen,et al.  Estimating censored regression models in the presence of nonparametric multiplicative heteroskedasticity , 2000 .

[15]  Xiaoming Zhang,et al.  An improved ordinal regression approach with Sum-of-Margin principle , 2010, 2010 Sixth International Conference on Natural Computation.

[16]  Ambuj Tewari,et al.  Sparseness vs Estimating Conditional Probabilities: Some Asymptotic Results , 2007, J. Mach. Learn. Res..

[17]  Gregory Kordas Smoothed binary regression quantiles , 2006 .

[18]  P. McCullagh Regression Models for Ordinal Data , 1980 .

[19]  Xiaoming Zhang,et al.  Kernel Discriminant Learning for Ordinal Regression , 2010, IEEE Transactions on Knowledge and Data Engineering.

[20]  Tsutomu Hirao,et al.  Order SVM: a kernel method for order learning based on generalized order statistics , 2005 .