Performance evaluation of three-objective genetic rule selection

We examine the classification performance of fuzzy rule-based systems designed by three-objective genetic rule selection. While a single rule set is usually obtained from a single run of rule generation methods, multiple rule sets are simultaneously obtained by a single run of our rule selection method with three objectives: to maximize the number of correctly classified training patterns, to minimize the number of selected fuzzy rules, and to minimize the total rule length. Our genetic rule selection is a two-stage approach. In the first stage, a pre-specified number of candidate fuzzy rules are extracted in a heuristic manner using a data mining technique. In the second stage, a multiobjective genetic algorithm is used for finding nondominated rule sets with respect to the three objectives. Since the first objective is measured on training patterns, the evolution of rule sets tends to overfit to training patterns. The question is whether the other two objectives work as a safeguard against the overfitting. In this paper, we examine the effect or the three-objective formulation on the generalization ability of obtained non-dominated rule sets. We also examine the effect of the adjustment of rule weights, which is performed after three-objective genetic rule selection.