Ordered incremental training for GA-based classifiers

This paper proposes ordered incremental genetic algorithms (OIGAs) to address the incremental training of input attributes for GA-based classifiers. Rather than learning attributes in batch as with normal GAs, OIGAs learn attributes one after another. Attributes are first arranged in different orders by evaluating their individual discriminating ability. Classification rule sets are then evolved incrementally to accommodate attributes continuously. By experimenting with different attribute orders, different approaches of OIGAs are evaluated using benchmark classification data sets. The simulation results show that OIGAs can achieve generally better performance than normal GAs, and OIGAs perform the best when training with a descending order of attributes.

[1]  L. Su,et al.  Incremental Self-Growing Neural Networks with the Changing Environment , 2001 .

[2]  Kenneth DeJong,et al.  Learning with genetic algorithms: An overview , 1988, Machine Learning.

[3]  Stewart W. Wilson,et al.  Learning Classifier Systems, From Foundations to Applications , 2000 .

[4]  Steven Guan,et al.  Incremental learning of collaborative classifier agents with new class acquisition: An incremental genetic algorithm approach , 2003, Int. J. Intell. Syst..

[5]  Vasant Honavar,et al.  Learn++: an incremental learning algorithm for supervised neural networks , 2001, IEEE Trans. Syst. Man Cybern. Part C.

[6]  Fangming Zhu,et al.  Class decomposition for GA-based classifier agents - a Pitt approach , 2004, IEEE Trans. Syst. Man Cybern. Part B.

[7]  Fengzhan Tian,et al.  Incremental learning of Bayesian networks with hidden variables , 2001, Proceedings 2001 IEEE International Conference on Data Mining.

[8]  Jürgen Schmidhuber,et al.  Optimal Ordered Problem Solver , 2002, Machine Learning.

[9]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[10]  Alex A. Freitas,et al.  Discovering comprehensible classification rules with a genetic algorithm , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[11]  John J. Grefenstette,et al.  Using Genetic Algorithms for Concept Learning , 2004, Machine Learning.

[12]  Zbigniew Michalewicz,et al.  Genetic algorithms + data structures = evolution programs (3rd ed.) , 1996 .

[13]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[14]  Vasant Honavar,et al.  Advances in the Evolutionary Synthesis of Intelligent Agents , 2001 .

[15]  Stephen F. Smith,et al.  A learning system based on genetic adaptive algorithms , 1980 .

[16]  Sandip Sen,et al.  Using real-valued genetic algorithms to evolve rule sets for classification , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[17]  Steven Guan,et al.  Incremental Learning with Respect to New Incoming Input Attributes , 2004, Neural Processing Letters.

[18]  John H. Holland,et al.  Escaping brittleness: the possibilities of general-purpose learning algorithms applied to parallel rule-based systems , 1995 .

[19]  John R. Anderson,et al.  MACHINE LEARNING An Artificial Intelligence Approach , 2009 .

[20]  Xavier Llorà,et al.  XCS and GALE: A Comparative Study of Two Learning Classifier Systems on Data Mining , 2001, IWLCS.

[21]  Vasant Honavar,et al.  Optimization of Classifiers Using Genetic Algorithms , 2001 .

[22]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[23]  Kishan G. Mehrotra,et al.  Efficient classification for multiclass problems using modular neural networks , 1995, IEEE Trans. Neural Networks.

[24]  Naohiro Ishii,et al.  Incremental learning methods with retrieving of interfered patterns , 1999, IEEE Trans. Neural Networks.

[25]  Hisao Ishibuchi,et al.  Performance evaluation of fuzzy classifier systems for multidimensional pattern classification problems , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[26]  Magne Setnes,et al.  GA-fuzzy modeling and classification: complexity and performance , 2000, IEEE Trans. Fuzzy Syst..

[27]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1996, Springer Berlin Heidelberg.

[28]  José Carlos Príncipe,et al.  Incremental backpropagation learning networks , 1996, IEEE Trans. Neural Networks.

[29]  John J. Grefenstette,et al.  Incremental Learning of Control Strategies with Genetic algorithms , 1989, ML.