A class decomposition approach for GA-based classifiers

Genetic algorithm (GA) has been used as a conventional method for classifiers to evolve solutions adaptively for classification problems. In this paper, a new approach using class decomposition is proposed to improve the performance of GA-based classifiers. A classification problem is fully partitioned into several class modules in the output domain and each module is responsible for solving a fraction of the original problem. These modules are trained in parallel and independently and the results obtained are integrated and evolved further for a final solution. A scheme based on Fisher's linear discriminant (FLD) computation is used to estimate the difficulty of separating two classes. Based on the FLD information derived, different integration approaches are proposed and their performance is compared. The experiment results on a benchmark data set show that class decomposition can achieve higher classification rate than the normal GA and FLD-based integration improves the classification accuracy further.

[1]  L. Su,et al.  Incremental Self-Growing Neural Networks with the Changing Environment , 2001 .

[2]  David G. Stork,et al.  Pattern Classification , 1973 .

[3]  Clifford Lau.  In memoriam: walter j. karplus , 2002, IEEE Trans. Neural Networks.

[4]  Lior Rokach,et al.  Improving Supervised Learning by Feature Decomposition , 2002, FoIKS.

[5]  Eric Michielssen,et al.  The use of domain decomposition genetic algorithms exploiting model reduction for the design of frequency selective surfaces , 2000 .

[6]  Steven Guan,et al.  Incremental Learning with Respect to New Incoming Input Attributes , 2004, Neural Processing Letters.

[7]  Steven Guan,et al.  Parallel growing and training of neural networks using output parallelism , 2002, IEEE Trans. Neural Networks.

[8]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[9]  Vasant Honavar,et al.  Advances in the Evolutionary Synthesis of Intelligent Agents , 2001 .

[10]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[11]  Edwin P. D. Pednault,et al.  Decomposition of Heterogeneous Classification Problems , 1997, IDA.

[12]  Giorgio Valentini,et al.  Parallel non-linear dichotomizers , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[13]  Casimir A. Kulikowski,et al.  Computer Systems That Learn: Classification and Prediction Methods from Statistics, Neural Nets, Machine Learning and Expert Systems , 1990 .

[14]  Magne Setnes,et al.  GA-fuzzy modeling and classification: complexity and performance , 2000, IEEE Trans. Fuzzy Syst..

[15]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1996, Springer Berlin Heidelberg.

[16]  Donald Michie,et al.  Problem Decomposition and the Learning of Skills , 1995, ECML.

[17]  Masami Ito,et al.  Task decomposition and module combination based on class relations: a modular neural network for pattern classification , 1999, IEEE Trans. Neural Networks.

[18]  Zbigniew Michalewicz,et al.  Genetic algorithms + data structures = evolution programs (2nd, extended ed.) , 1994 .

[19]  Vasant Honavar,et al.  Optimization of Classifiers Using Genetic Algorithms , 2001 .

[20]  Pavel Brazdil,et al.  Proceedings of the European Conference on Machine Learning , 1993 .

[21]  Sandip Sen,et al.  Using real-valued genetic algorithms to evolve rule sets for classification , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[22]  Kishan G. Mehrotra,et al.  Efficient classification for multiclass problems using modular neural networks , 1995, IEEE Trans. Neural Networks.

[23]  Steven Guan,et al.  Incremental learning of collaborative classifier agents with new class acquisition: An incremental genetic algorithm approach , 2003, Int. J. Intell. Syst..

[24]  Robert E. Jenkins,et al.  A simplified neural network solution through problem decomposition: the case of the truck backer-upper , 1993, IEEE Trans. Neural Networks.

[25]  Hisao Ishibuchi,et al.  Performance evaluation of fuzzy classifier systems for multidimensional pattern classification problems , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[26]  Kenneth A. De Jong,et al.  Learning Concept Classification Rules Using Genetic Algorithms , 1991, IJCAI.