Learning Classifier System Ensembles With Rule-Sharing

This paper presents an investigation into exploiting the population-based nature of learning classifier systems (LCSs) for their use within highly parallel systems. In particular, the use of simple payoff and accuracy-based LCSs within the ensemble machine approach is examined. Results indicate that inclusion of a rule migration mechanism inspired by parallel genetic algorithms is an effective way to improve learning speed in comparison to equivalent single systems. Presentation of a mechanism which exploits the underlying niche-based generalization mechanism of accuracy-based systems is then shown to further improve their performance, particularly, as task complexity increases. This is not found to be the case for payoff-based systems. Finally, considerably better than linear speedup is demonstrated with the accuracy-based systems on a version of the well-known Boolean logic benchmark task used throughout.

[1]  Sargur N. Srihari,et al.  Decision Combination in Multiple Classifier Systems , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Alex Alves Freitas,et al.  Mining Very Large Databases with Parallel Processing , 1997, The Kluwer International Series on Advances in Database Systems.

[3]  Larry Bull,et al.  Towards distributed adaptive control for road traffic junction signals using learning classifier systems , 2004 .

[4]  Larry Bull,et al.  Two simple learning classifier systems , 2005 .

[5]  Larry Bull,et al.  Learning Classifier Systems , 2002, Annual Conference on Genetic and Evolutionary Computation.

[6]  A. Greenyer Coil 2000 Competition The use of a learning classifier system JXCS , 2001 .

[7]  Rakesh Agrawal,et al.  SPRINT: A Scalable Parallel Classifier for Data Mining , 1996, VLDB.

[8]  Larry Bull,et al.  On using ZCS in a Simulated Continuous Double-Auction Market , 1999, GECCO.

[9]  George G. Robertson,et al.  A tale of two classifier systems , 1988, Machine Learning.

[10]  Filippo Neri,et al.  A Parallel Genetic Algorithm for Concept Learning , 1995, ICGA.

[11]  Chandrika Kamath,et al.  Inducing oblique decision trees with evolutionary algorithms , 2003, IEEE Trans. Evol. Comput..

[12]  F. Serendynski,et al.  Learning classifier systems in multi-agent environments , 1995 .

[13]  J. Holland,et al.  Artificial Adaptive Agents in Economic Theory , 1991 .

[14]  Bernard Manderick,et al.  Fine-Grained Parallel Genetic Algorithms , 1989, ICGA.

[15]  Mohammed J. Zaki,et al.  Large-Scale Parallel Data Mining , 2002, Lecture Notes in Computer Science.

[16]  Tim Kovacs,et al.  Towards a Theory of Strong Overgeneral Classifiers , 2000, FOGA.

[17]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[18]  Thomas G. Dietterich An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization , 2000, Machine Learning.

[19]  Larry Bull,et al.  On accuracy-based fitness , 2002, Soft Comput..

[20]  Anthony J. Bagnall,et al.  Using an Adaptive Agent to Bid in a Simplified Model of the UK Market in Electricity , 1999, Annual Conference on Genetic and Evolutionary Computation.

[21]  Stewart W. Wilson Classifier Fitness Based on Accuracy , 1995, Evolutionary Computation.

[22]  Martin V. Butz,et al.  Toward a theory of generalization and learning in XCS , 2004, IEEE Transactions on Evolutionary Computation.

[23]  Aiko M. Hormann,et al.  Programs for Machine Learning. Part I , 1962, Inf. Control..

[24]  Dana S. Richards,et al.  Punctuated Equilibria: A Parallel Genetic Algorithm , 1987, ICGA.

[25]  Terence C. Fogarty,et al.  Co-evolutionary classifier systems for multi-agent simulation , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[26]  Ellen R. McGrattan,et al.  Money as a medium of exchange in an economy with artificially intelligent agents , 1990 .

[27]  Larry Bull,et al.  Applications of Learning Classifier Systems , 2004 .

[28]  Xin Yao,et al.  Diversity creation methods: a survey and categorisation , 2004, Inf. Fusion.

[29]  Stephen F. Smith,et al.  A learning system based on genetic adaptive algorithms , 1980 .

[30]  Erick Cantú-Paz,et al.  Efficient and Accurate Parallel Genetic Algorithms , 2000, Genetic Algorithms and Evolutionary Computation.

[31]  Ludmila I. Kuncheva,et al.  Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy , 2003, Machine Learning.

[32]  R. Palmer,et al.  Artificial economic life: a simple model of a stockmarket , 1994 .

[33]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[34]  Alexandre Parodi,et al.  An Efficient Classifier System and Its Experimental Comparison with Two Representative Learning Methods on Three Medical Domains , 1991, ICGA.

[35]  Stewart W. Wilson Mining Oblique Data with XCS , 2000, IWLCS.

[36]  Giandomenico Spezzano,et al.  Genetic Programming and Simulated Annealing: A Hybrid Method to Evolve Decision Trees , 2000, EuroGP.