Cost-Sensitive Universum-SVM

Many applications of machine learning involve analysis of sparse high-dimensional data, where the number of input features is larger than the number of data samples. Standard classification methods may not be sufficient for such data, and this provides motivation for non-standard learning settings. One such new learning methodology is called Learning through Contradictions or Universum support vector machine (U-SVM) [1, 2]. Recent studies [2-10] have shown U-SVM to be quite effective for such sparse high-dimensional data settings. However, these studies use balanced data sets with equal misclassification costs. This paper extends the U-SVM for problems with different misclassification costs, and presents practical conditions for the effectiveness of the cost sensitive U-SVM. Finally, several empirical comparisons are presented to illustrate the utility of the proposed approach.

[1]  Vladimir Cherkassky,et al.  Learning from Data: Concepts, Theory, and Methods , 1998 .

[2]  Charles Elkan,et al.  The Foundations of Cost-Sensitive Learning , 2001, IJCAI.

[3]  Johannes Stallkamp,et al.  The German Traffic Sign Recognition Benchmark: A multi-class classification competition , 2011, The 2011 International Joint Conference on Neural Networks.

[4]  Vladimir Cherkassky,et al.  Gender classification of human faces using inference through contradictions , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[5]  Gary M. Weiss,et al.  Cost-Sensitive Learning vs. Sampling: Which is Best for Handling Unbalanced Classes with Unequal Error Costs? , 2007, DMIN.

[6]  Jason Weston,et al.  Inference with the Universum , 2006, ICML.

[7]  Daniel T. Larose,et al.  Discovering Knowledge in Data: An Introduction to Data Mining , 2005 .

[8]  Changshui Zhang,et al.  Selecting Informative Universum Sample for Semi-Supervised Learning , 2009, IJCAI.

[9]  Yi Lin,et al.  Support Vector Machines for Classification in Nonstandard Situations , 2002, Machine Learning.

[10]  Robert A. Lordo,et al.  Learning from Data: Concepts, Theory, and Methods , 2001, Technometrics.

[11]  Wuyang Dai,et al.  Empirical Study of the Universum SVM Learning for High-Dimensional Data , 2009, ICANN.

[12]  Jing Ling,et al.  On Universum-Support Vector Machines , 2009 .

[13]  Fei Wang,et al.  Semi-Supervised Classification with Universum , 2008, SDM.

[14]  Vladimir Cherkassky,et al.  Simple Method for Interpretation of High-Dimensional Nonlinear SVM Classification Models , 2010, DMIN.

[15]  Ronald A. Cole,et al.  Spoken Letter Recognition , 1990, HLT.

[16]  Bernhard Schölkopf,et al.  An Analysis of Inference with the Universum , 2007, NIPS.

[17]  Vladimir Naumovich Vapnik,et al.  Estimation of dependences based on empirical data ; : Empirical inference science : afterword of 2006 , 2006 .

[18]  Wuyang Dai,et al.  Practical Conditions for Effectiveness of the Universum Learning , 2011, IEEE Transactions on Neural Networks.