Adaptive Metric Learning Vector Quantization for Ordinal Classification

Many pattern analysis problems require classification of examples into naturally ordered classes. In such cases, nominal classification schemes will ignore the class order relationships, which can have a detrimental effect on classification accuracy. This article introduces two novel ordinal learning vector quantization (LVQ) schemes, with metric learning, specifically designed for classifying data items into ordered classes. In ordinal LVQ, unlike in nominal LVQ, the class order information is used during training in selecting the class prototypes to be adapted, as well as in determining the exact manner in which the prototypes get updated. Prototype-based models in general are more amenable to interpretations and can often be constructed at a smaller computational cost than alternative nonlinear classification models. Experiments demonstrate that the proposed ordinal LVQ formulations compare favorably with their nominal counterparts. Moreover, our methods achieve competitive performance against existing benchmark ordinal regression models.

[1]  Ling Li,et al.  Ordinal Regression by Extended Binary Classification , 2006, NIPS.

[2]  Jaime S. Cardoso,et al.  Modelling ordinal relations with SVMs: An application to objective aesthetic evaluation of breast cancer conservative treatment , 2005, Neural Networks.

[3]  Christopher J. Merz,et al.  UCI Repository of Machine Learning Databases , 1996 .

[4]  Wojciech Kotlowski,et al.  Ordinal Classification with Decision Rules , 2007, MCD.

[5]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[6]  Michael Biehl,et al.  Adaptive Relevance Matrices in Learning Vector Quantization , 2009, Neural Computation.

[7]  Atsushi Sato,et al.  Generalized Learning Vector Quantization , 1995, NIPS.

[8]  Willem Waegeman,et al.  An ensemble of Weighted Support Vector Machines for Ordinal Regression , 2007 .

[9]  Andrea Esuli,et al.  Evaluation Measures for Ordinal Regression , 2009, 2009 Ninth International Conference on Intelligent Systems Design and Applications.

[10]  John Moody,et al.  Learning rate schedules for faster stochastic gradient search , 1992, Neural Networks for Signal Processing II Proceedings of the 1992 IEEE Workshop.

[11]  Petra Schneider.  Advanced methods for prototype-based classification , 2010 .

[12]  Amnon Shashua,et al.  Ranking with Large Margin Principle: Two Approaches , 2002, NIPS.

[13]  Fen Xia,et al.  Ordinal Regression as Multiclass Classification , 2007 .

[14]  Wei Chu,et al.  Support Vector Ordinal Regression , 2007, Neural Computation.

[15]  Eibe Frank,et al.  A Simple Approach to Ordinal Classification , 2001, ECML.

[16]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[17]  Thomas Villmann,et al.  On the Generalization Ability of GRLVQ Networks , 2005, Neural Processing Letters.

[18]  Teuvo Kohonen,et al.  Learning vector quantization , 1998 .

[19]  Xiaoming Zhang,et al.  Kernel Discriminant Learning for Ordinal Regression , 2010, IEEE Transactions on Knowledge and Data Engineering.

[20]  Thomas Villmann,et al.  Generalized relevance learning vector quantization , 2002, Neural Networks.

[21]  Ling Li,et al.  Reduction from Cost-Sensitive Ordinal Ranking to Weighted Binary Classification , 2012, Neural Computation.