Cost-Sensitive Classification: Empirical Evaluation of a Hybrid Genetic Decision Tree Induction Algorithm

This paper introduces ICET, a new algorithm for cost-sensitive classification. ICET uses a genetic algorithm to evolve a population of biases for a decision tree induction algorithm. The fitness function of the genetic algorithm is the average cost of classification when using the decision tree, including both the costs of tests (features, measurements) and the costs of classification errors. ICET is compared here with three other algorithms for cost-sensitive classification -- EG2, CS-ID3, and IDX -- and also with C4.5, which classifies without regard to cost. The five algorithms are evaluated empirically on five real-world medical datasets. Three sets of experiments are performed. The first set examines the baseline performance of the five algorithms on the five datasets and establishes that ICET performs significantly better than its competitors. The second set tests the robustness of ICET under a variety of conditions and shows that ICET maintains its advantage. The third set looks at ICET's search in bias space and discovers a way to improve the search.

[1]  J. Baldwin.  A New Factor in Evolution , 1896, The American Naturalist.

[2]  C L Morgan,et al.  ON MODIFICATION AND VARIATION. , 1896, Science.

[3]  C. Waddington.  Canalization of Development and the Inheritance of Acquired Characters , 1942, Nature.

[4]  J. Habbema,et al.  Cases of doubt in allocation problems , 1974 .

[5]  J. Friedman,et al.  Projection Pursuit Regression , 1981 .

[6]  Judea Pearl,et al.  Heuristics : intelligent search strategies for computer problem solving , 1984 .

[7]  John J. Grefenstette,et al.  Optimization of Control Parameters for Genetic Algorithms , 1986, IEEE Transactions on Systems, Man, and Cybernetics.

[8]  John Maynard Smith,et al.  When learning guides evolution , 1987, Nature.

[9]  Geoffrey E. Hinton,et al.  How Learning Can Guide Evolution , 1996, Complex Syst..

[10]  Marlon Núñez,et al.  Economic Induction: A Case Study , 1988, EWSL.

[11]  Donald Perlis,et al.  Explicitly biased generalization , 1989, Comput. Intell..

[12]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[13]  Ming Tan,et al.  Cost-Sensitive Concept Learning of Sensor Use in Approach ad Recognition , 1989, ML.

[14]  Steven W. Norton.  Generating Better Decision Trees , 1989, IJCAI.

[15]  Larry A. Rendell,et al.  Building Robust Learning Systems by Combining Induction and Optimization , 1989, IJCAI.

[16]  Ming Tan,et al.  CSL: a cost-sensitive learning system for sensing and grasping objects , 1990, Proceedings., IEEE International Conference on Robotics and Automation.

[17]  Floor Verdenius,et al.  A Method for Inductive Cost Optimization , 1991, EWSL.

[18]  William M. Spears,et al.  An Artificial Intelligence Approach to Analog Systems Diagnosis , 1991 .

[19]  David H. Ackley,et al.  Interactions between learning and evolution , 1991 .

[20]  William M. Spears,et al.  Crossover or Mutation? , 1992, FOGA.

[21]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[22]  J. D. Schaffer,et al.  Combinations of genetic algorithms and neural networks: a survey of the state of the art , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[23]  Larry A. Rendell,et al.  Lookahead Feature Construction for Learning Hard Concepts , 1993, International Conference on Machine Learning.

[24]  L. Darrell Whitley,et al.  Adding Learning to the Cellular Development of Neural Networks: Evolution and the Baldwin Effect , 1993, Evolutionary Computation.

[25]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[26]  Ron Rymon.  An SE-tree based Characterization of the Induction Problem , 1993, ICML.

[27]  Arno Sprecher,et al.  An Artificial Intelligence Approach , 1994 .

[28]  L. Darrell Whitley,et al.  Lamarckian Evolution, The Baldwin Effect and Function Optimization , 1994, PPSN.

[29]  Gholamreza Nakhaeizadeh,et al.  Cost-Sensitive Pruning of Decision Trees , 1994, ECML.

[30]  David W. Aha,et al.  Feature Selection for Case-Based Classification of Cloud Types: An Empirical Comparison , 1994 .

[31]  Foster Provost,et al.  Goal-Directed Inductive Learning: Trading off Accuracy for Reduced Error Cost , 1994 .

[32]  R. Anderson,et al.  Learning and evolution: a quantitative genetics approach. , 1995, Journal of theoretical biology.