Cost-sensitive classification with genetic programming

Cost-sensitive classification is an attractive topic in data mining. Although genetic programming (GP) technique has been applied to general classification, to our knowledge, it has not been exploited to address cost-sensitive classification in the literature, where the costs of misclassification errors are non-uniform. To investigate the applicability of GP to cost-sensitive classification, this paper first reviews the existing methods of cost-sensitive classification in data mining. We then apply GP to address cost-sensitive classification by means of two methods through: a) manipulating training data, and b) modifying the learning algorithm. In particular, a constrained genetic programming (CGP), a GP-based cost-sensitive classifier, has been introduced in this study. CGP is capable of building decision trees to minimize not only the expected number of errors, but also the expected misclassification costs through a novel constraint fitness function. CGP has been tested on the heart disease dataset and the German credit dataset from the UCI repository. Its efficacy with respect to cost has been demonstrated by comparisons with non-cost-sensitive learning methods and cost-sensitive learning methods in terms of the costs.

[1]  M. F.,et al.  Bibliography , 1985, Experimental Gerontology.

[2]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[3]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[4]  Christopher R. Houck,et al.  On the use of non-stationary penalty functions to solve nonlinear constrained optimization problems with GA's , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[5]  David J. Spiegelhalter,et al.  Machine Learning, Neural and Statistical Classification , 2009 .

[6]  J. K. Kinnear,et al.  Advances in Genetic Programming , 1994 .

[7]  Peter D. Turney.  Cost-Sensitive Classification: Empirical Evaluation of a Hybrid Genetic Decision Tree Induction Algorithm , 1994, J. Artif. Intell. Res..

[8]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[9]  J. Ross Quinlan,et al.  Bagging, Boosting, and C4.5 , 1996, AAAI/IAAI, Vol. 1.

[10]  Geoffrey I. Webb.  Cost-Sensitive Specialization , 1996, PRICAI.

[11]  Zbigniew Michalewicz,et al.  Evolutionary Algorithms for Constrained Parameter Optimization Problems , 1996, Evolutionary Computation.

[12]  Bastien Chopard,et al.  Parallel Genetic Programming and its Application to Trading Model Induction , 1997, Parallel Comput..

[13]  Kai Ming Ting,et al.  Boosting Cost-Sensitive Trees , 1998, Discovery Science.

[14]  Carla E. Brodley,et al.  Pruning Decision Trees with Misclassification Costs , 1998, ECML.

[15]  Salvatore J. Stolfo,et al.  Toward Scalable Learning with Non-Uniform Class and Cost Distributions: A Case Study in Credit Card Fraud Detection , 1998, KDD.

[16]  Ian H. Witten,et al.  Generating Accurate Rule Sets Without Global Optimization , 1998, ICML.

[17]  Igor Kononenko,et al.  Cost-Sensitive Learning with Neural Networks , 1998, ECAI.

[18]  Edward Tsang,et al.  Investment decision making using FGP: a case study , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[19]  Pedro M. Domingos.  MetaCost: a general method for making classifiers cost-sensitive , 1999, KDD '99.

[20]  Salvatore J. Stolfo,et al.  AdaCost: Misclassification Cost-Sensitive Boosting , 1999, ICML.

[21]  E. Tsang,et al.  Reducing Failures In Investment Recommendations Using Genetic Programming , 2000 .

[22]  Kai Ming Ting,et al.  An Instance-weighting Method to Induce Cost-sensitive Trees , 2001 .

[23]  Peter D. Turney.  Types of Cost in Inductive Concept Learning , 2002, ArXiv.

[24]  Thomas G. Dietterich,et al.  Methods for cost-sensitive learning , 2002 .

[25]  Edward Tsang,et al.  Eddie for Financial Forecasting , 2002 .

[26]  Dr. Alex A. Freitas.  Data Mining and Knowledge Discovery with Evolutionary Algorithms , 2002, Natural Computing Series.

[27]  Xin Yao,et al.  A novel evolutionary data mining algorithm with applications to churn prediction , 2003, IEEE Trans. Evol. Comput..

[28]  Alex A. Freitas,et al.  A survey of evolutionary algorithms for data mining and knowledge discovery , 2003 .

[29]  Alex Alves Freitas,et al.  Guest editorial data mining and knowledge discovery with evolutionary algorithms , 2003, IEEE Trans. Evol. Comput..

[30]  Foster J. Provost,et al.  Inductive policy: The pragmatics of bias selection , 1995, Machine Learning.

[31]  Tom Fawcett,et al.  Adaptive Fraud Detection , 1997, Data Mining and Knowledge Discovery.

[32]  Eric Bauer,et al.  An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants , 1999, Machine Learning.