An incremental approach to genetic-algorithms-based classification

Incremental learning has been widely addressed in the machine learning literature to cope with learning tasks where the learning environment is ever changing or training samples become available over time. However, most research work explores incremental learning with statistical algorithms or neural networks, rather than evolutionary algorithms. The work in this paper employs genetic algorithms (GAs) as basic learning algorithms for incremental learning within one or more classifier agents in a multiagent environment. Four new approaches with different initialization schemes are proposed. They keep the old solutions and use an "integration" operation to integrate them with new elements to accommodate new attributes, while biased mutation and crossover operations are adopted to further evolve a reinforced solution. The simulation results on benchmark classification data sets show that the proposed approaches can deal with the arrival of new input attributes and integrate them with the original input space. It is also shown that the proposed approaches can be successfully used for incremental learning and improve classification rates as compared to the retraining GA. Possible applications for continuous incremental training and feature selection are also discussed.

[1]  Naohiro Ishii,et al.  Incremental learning methods with retrieving of interfered patterns , 1999, IEEE Trans. Neural Networks.

[2]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[3]  K. D. Jong Learning with Genetic Algorithms: An Overview , 2005, Machine Learning.

[4]  R. Brits,et al.  A clustering approach to incremental learning for feedforward neural networks , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[5]  Stewart W. Wilson,et al.  Learning Classifier Systems, From Foundations to Applications , 2000 .

[6]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[7]  Christophe G. Giraud-Carrier,et al.  A Note on the Utility of Incremental Learning , 2000, AI Commun..

[8]  Vasant Honavar,et al.  Learn++: an incremental learning algorithm for supervised neural networks , 2001, IEEE Trans. Syst. Man Cybern. Part C.

[9]  Günter Rudolph,et al.  Contemporary Evolution Strategies , 1995, ECAL.

[10]  J. J. Merelo Optimization of Classifiers Using Genetic Algorithms , 1996 .

[11]  Kenneth A. De Jong,et al.  Learning Concept Classification Rules Using Genetic Algorithms , 1991, IJCAI.

[12]  José Carlos Príncipe,et al.  Incremental backpropagation learning networks , 1996, IEEE Trans. Neural Networks.

[13]  Zbigniew Michalewicz,et al.  Genetic algorithms + data structures = evolution programs (3rd ed.) , 1996 .

[14]  Christopher J. Merz,et al.  UCI Repository of Machine Learning Databases , 1996 .

[15]  L. Su,et al.  Incremental Self-Growing Neural Networks with the Changing Environment , 2001 .

[16]  S. K. Chang What is a software agent ? , 2002 .

[17]  C. Escazut,et al.  Classifier systems evolving multi-agent system with distributed elitism , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[18]  David B. Fogel,et al.  Meta-evolutionary programming , 1991, [1991] Conference Record of the Twenty-Fifth Asilomar Conference on Signals, Systems & Computers.

[19]  Stephen F. Smith,et al.  A learning system based on genetic adaptive algorithms , 1980 .

[20]  Liva Ralaivola,et al.  Incremental Learning Algorithms for Classification and Regression: local strategies , 2002 .

[21]  Sandip Sen,et al.  Using real-valued genetic algorithms to evolve rule sets for classification , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[22]  John H. Holland,et al.  Escaping brittleness: the possibilities of general-purpose learning algorithms applied to parallel rule-based systems , 1995 .

[23]  Vidroha Debroy,et al.  Genetic Programming , 1998, Lecture Notes in Computer Science.

[24]  Magne Setnes,et al.  GA-fuzzy modeling and classification: complexity and performance , 2000, IEEE Trans. Fuzzy Syst..

[25]  Hisao Ishibuchi,et al.  Improving the performance of fuzzy classifier systems for pattern classification problems with continuous attributes , 1999, IEEE Trans. Ind. Electron..

[26]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1996, Springer Berlin Heidelberg.

[27]  Casimir A. Kulikowski,et al.  Computer Systems That Learn: Classification and Prediction Methods from Statistics, Neural Nets, Machine Learning and Expert Systems , 1990 .

[28]  Fernando Santos Osório,et al.  INSS: A hybrid system for constructive machine learning , 1999, Neurocomputing.

[29]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[30]  D. Goldberg,et al.  Linkage learning through probabilistic expression , 2000 .

[31]  C. Giraud-Carrier,et al.  A Constructive Incremental Learning Algorithm for Binary Classification Tasks , 2006, 2006 IEEE Mountain Workshop on Adaptive and Learning Systems.

[32]  Steven Guan,et al.  Incremental Learning with Respect to New Incoming Input Attributes , 2004, Neural Processing Letters.

[33]  Hoon Kang,et al.  Evolvable cellular classifiers , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[34]  Steven Guan,et al.  Ordered incremental training with genetic algorithms , 2004, Int. J. Intell. Syst..

[35]  Hisao Ishibuchi,et al.  Performance evaluation of fuzzy classifier systems for multidimensional pattern classification problems , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[36]  Xavier Llorà,et al.  XCS and GALE: A Comparative Study of Two Learning Classifier Systems on Data Mining , 2001, IWLCS.

[37]  Vasant Honavar,et al.  Optimization of Classifiers Using Genetic Algorithms , 2001 .

[38]  Lutz Prechelt,et al.  A Set of Neural Network Benchmark Problems and Benchmarking Rules , 1994 .

[39]  D. M. Hutton,et al.  Advances in the Evolutionary Synthesis of Intelligent Agents , 2002 .

[40]  David E. Goldberg,et al.  Linkage Problem, Distribution Estimation, and Bayesian Networks , 2000, Evolutionary Computation.