From circular ordinal regression to multilabel classification

Several applications domains like wind forecasting in meteorology and robot control in robotics demand for learning algorithms that are able to make discrete directional predictions. We refer to this problem setting as circular ordinal regression, since it shares the same properties as traditional ordinal regression, namely the need for a specific model structure and order-preserving loss functions. This article gives a detailed introduction to the topic and proposes two methods. The first one is a circular support vector approach (cSVM), parameterized with only two vectors. The second method converts circular ordinal regression to a multilabel classification approach that takes the circular ranking into account by minimizing the Hamming loss. We also present initial empirical results based on two toy examples and a real-life application in the area of brain-computer interfaces.

[1]  Tom M. Mitchell,et al.  Using the Future to Sort Out the Present: Rankprop and Multitask Learning for Medical Risk Evaluation , 1995, NIPS.

[2]  Bruno Bauwens,et al.  Directional predictions for 4-class BCI data , 2010, ESANN.

[3]  Gianluca Pollastri,et al.  A neural network approach to ordinal regression , 2007, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[4]  Eibe Frank,et al.  A Simple Approach to Ordinal Classification , 2001, ECML.

[5]  Neil D. Lawrence,et al.  Advances in Neural Information Processing Systems 14 , 2002 .

[6]  Eyke Hüllermeier,et al.  Bayes Optimal Multilabel Classification via Probabilistic Classifier Chains , 2010, ICML.

[7]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[8]  G. Pfurtscheller,et al.  Designing optimal spatial filters for single-trial EEG classification in a movement task , 1999, Clinical Neurophysiology.

[9]  Eyke Hüllermeier,et al.  On label dependence in multilabel classification , 2010, ICML 2010.

[10]  Ling Li,et al.  Ordinal Regression by Extended Binary Classification , 2006, NIPS.

[11]  Bernard De Baets,et al.  ROC analysis in ordinal regression learning , 2008, Pattern Recognit. Lett..

[12]  Eyke Hüllermeier,et al.  Regret Analysis for Performance Metrics in Multi-Label Classification: The Case of Hamming and Subset Zero-One Loss , 2010, ECML/PKDD.

[13]  Wei Chu,et al.  Support Vector Ordinal Regression , 2007, Neural Computation.

[14]  Koby Crammer,et al.  Pranking with Ranking , 2001, NIPS.

[15]  Amnon Shashua,et al.  Ranking with Large Margin Principle: Two Approaches , 2002, NIPS.