Evolutionary Multiobjective Knowledge Extraction for High-Dimensional Pattern Classification Problems

This paper proposes an evolutionary multiobjective optimization (EMO) approach to knowledge extraction from numerical data for high-dimensional pattern classification problems with many continuous attributes. The proposed approach is a three-stage rule extraction method. First each continuous attribute is discretized into several intervals using a class entropy measure. In this stage, multiple partitions with different granularity are specified. Next a prespecified number of candidate rules are generated from numerical data using a heuristic rule evaluation measure in a similar manner to data mining. Then a small number of candidate rules are selected by an EMO algorithm. The EMO algorithm tries to maximize the accuracy of selected rules. At the same time, it tries to minimize their complexity. Our rule selection problem has three objectives: to maximize the number of correctly classified training patterns, to minimize the number of selected rules and to minimize their total rule length. The length of each rule is defined by the number of its antecedent conditions. The main characteristic feature of the proposed EMO approach is that many rule sets with different accuracy and different complexity are simultaneously obtained from its single run. They are tradeoff solutions (i.e., non-dominated rule sets) with respect to the accuracy and the complexity. Through computational experiments, we demonstrate the applicability of the proposed EMO approach to high-dimensional pattern classification problems with many continuous attributes. We also demonstrate some advantages of the proposed EMO approach over single-objective ones.