Linear models for cost-sensitive classification

In this paper, we investigate the performance of statistical, mathematical programming and heuristic linear models for cost-sensitive classification. In particular, we use five cost-sensitive techniques including Fisher's discriminant analysis DA, asymmetric misclassification cost mixed integer programming AMC-MIP, cost-sensitive support vector machine CS-SVM, a hybrid support vector machine and mixed integer programming SVMIP and heuristic cost-sensitive genetic algorithm CGA techniques. Using simulated datasets of varying group overlaps, data distributions and class biases, and real-world datasets from financial and medical domains, we compare the performances of our five techniques based on overall holdout sample misclassification cost. The results of our experiments on simulated datasets indicate that when group overlap is low and data distribution is exponential, DA appears to provide superior performance. For all other situations with simulated datasets, CS-SVM provides superior performance. In case of real-world datasets from financial domain, CGA and AMC-MIP hold a slight edge over the two SVM-based classifiers. However, for medical domains with mixed continuous and discrete attributes, SVM classifiers perform better than heuristic CGA and AMC-MIP classifiers. The SVMIP model is the most computationally inefficient model and poor performing model.

[1]  Zou Peng,et al.  The method for solving two types of errors in customer segmentation on unbalanced data , 2008, ICEC.

[2]  Vijay S. Mookerjee,et al.  Inductive Expert System Design: Maximizing System Value , 1993, Inf. Syst. Res..

[3]  Michel Benaroch,et al.  Adding Value to Induced Decision Trees for Time-Sensitive Data , 1997, INFORMS J. Comput..

[4]  Olivier Chapelle,et al.  Training a Support Vector Machine in the Primal , 2007, Neural Computation.

[5]  Peter D. Turney Cost-Sensitive Classification: Empirical Evaluation of a Hybrid Genetic Decision Tree Induction Algorithm , 1994, J. Artif. Intell. Res..

[6]  Fabio Roli,et al.  Cost-sensitive Learning in Support Vector Machines , 2002 .

[7]  Lazaros G. Papageorgiou,et al.  A mixed integer optimisation model for data classification , 2009, Comput. Ind. Eng..

[8]  Kwei Tang,et al.  Cost-Sensitive Decision Tree Induction with Label-Dependent Late Constraints , 2014, INFORMS J. Comput..

[9]  Ian Witten,et al.  Data Mining , 2000 .

[10]  John M. Liittschwager,et al.  Integer Programming Solution of a Classification Problem , 1978 .

[11]  Ulf Brefeld,et al.  Support Vector Machines with Example Dependent Costs , 2003, ECML.

[12]  Emilio Carrizosa,et al.  Two-group classification via a biobjective margin maximization model , 2006, Eur. J. Oper. Res..

[13]  Xiaoqing Ding,et al.  Face Detection Based on Cost-Sensitive Support Vector Machines , 2002, SVM.

[14]  S. M. Bajgier,et al.  AN EXPERIMENTAL COMPARISON OF STATISTICAL AND LINEAR PROGRAMMING APPROACHES TO THE DISCRIMINANT PROBLEM , 1982 .

[15]  Parag C. Pendharkar,et al.  DEA based dimensionality reduction for classification problems satisfying strict non-satiety assumption , 2011, Eur. J. Oper. Res..

[16]  P. Pendharkar,et al.  Misclassification cost minimizing fitness functions for genetic algorithm-based artificial neural network classifiers , 2009, J. Oper. Res. Soc..

[17]  T. W. Anderson,et al.  An Introduction to Multivariate Statistical Analysis , 1959 .

[18]  Frederick S. Hillier,et al.  Introduction to Operations Research and Revised CD-ROM 8 , 2005 .

[19]  Sudhir Nanda,et al.  A misclassification cost-minimizing evolutionary–neural classification approach , 2006 .

[20]  Paul A. Rubin,et al.  Heuristic solution procedures for a mixed‐integer programming discriminant model , 1990 .

[21]  Antonie Stam,et al.  A mixed integer programming algorithm for minimizing the training sample misclassification cost in two-group classification , 1997, Ann. Oper. Res..

[22]  Sudhir Nanda,et al.  Linear models for minimizing misclassification costs in bankruptcy prediction , 2001, Intell. Syst. Account. Finance Manag..

[23]  A. Stam,et al.  Classification performance of mathematical programming techniques in discriminant analysis: Results for small and medium sample sizes , 1990 .