A biobjective model to select features with good classification quality and low cost

In this paper we address a multigroup classification problem in which we want to take into account, together with the generalization ability, costs associated with the features. This cost is not limited to an economical payment, but can also refer to risk, computational effort, space requirements, etc. In order to get a good generalization ability, we use support vector machines (SVM) as the basic mechanism by considering the maximization of the margin. We formulate the problem as a biobjective mixed integer problem, for which Pareto optimal solutions can be obtained.

[1]  Ming Tan,et al.  Cost-sensitive learning of classification knowledge and its applications in robotics , 2004, Machine Learning.

[2]  Thomas G. Dietterich,et al.  Learning cost-sensitive diagnostic policies from data , 2003 .

[3]  Ayhan Demiriz,et al.  Linear Programming Boosting via Column Generation , 2002, Machine Learning.

[4]  Toshihide Ibaraki,et al.  Logical analysis of numerical data , 1997, Math. Program..

[5]  Jacques Teghem,et al.  Two-phases Method and Branch and Bound Procedures to Solve the Bi–objective Knapsack Problem , 1998, J. Glob. Optim..

[6]  James E. Falk,et al.  Robust separation of finite sets via quadratics , 2001, Comput. Oper. Res..

[7]  Robert Tibshirani,et al.  Classification by Pairwise Coupling , 1997, NIPS.

[8]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[9]  Sarah Cobey,et al.  Grants and Awards , 1981, Nursing in critical care.

[10]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[11]  Peter D. Turney Cost-Sensitive Classification: Empirical Evaluation of a Hybrid Genetic Decision Tree Induction Algorithm , 1994, J. Artif. Intell. Res..

[12]  Nello Cristianini,et al.  An introduction to Support Vector Machines , 2000 .

[13]  Yoram Singer,et al.  Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers , 2000, J. Mach. Learn. Res..

[14]  Steven W. Norton,et al.  Generating Better Decision Trees , 1989, IJCAI.

[15]  Antonie Stam,et al.  Second order mathematical programming formulations for discriminant analysis , 1994 .

[16]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.