Evolutionary Optimization Guided by Entropy-Based Discretization

The Learnable Evolution Model (LEM) involves alternating periods of optimization and learning, performa extremely well on a range of problems, a specialises in achieveing good results in relatively few function evaluations. LEM implementations tend to use sophisticated learning strategies. Here we continue an exploration of alternative and simpler learning strategies, and try Entropy-based Discretization (ED), whereby, for each parameter in the search space, we infer from recent evaluated samples what seems to be a `good' interval. We find that LEM(ED) provides significant advantages in both solution speed and quality over the unadorned evolutionary algorithm, and is usually superior to CMA-ES when the number of evaluations is limited. It is interesting to see such improvement gained from an easily-implemented approach. LEM(ED) can be tentatively recommended for trial on problems where good results are needed in relatively few fitness evaluations, while it is open to several routes of extension and further sophistication. Finally, results reported here are not based on a modern function optimization suite, but ongoing work confirms that our findings remain valid for non-separable functions.

[1]  Nikolaus Hansen,et al.  A restart CMA evolution strategy with increasing population size , 2005, 2005 IEEE Congress on Evolutionary Computation.

[2]  J. Ford,et al.  Hybrid estimation of distribution algorithm for global optimization , 2004 .

[3]  Pedro Larrañaga,et al.  Experimental Results in Function Optimization with EDAs in Continuous Domain , 2002, Estimation of Distribution Algorithms.

[4]  Moonis Ali,et al.  Innovations in Applied Artificial Intelligence , 2005 .

[5]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Comparing Review , 2006, Towards a New Evolutionary Computation.

[6]  Huan Liu,et al.  Discretization: An Enabling Technique , 2002, Data Mining and Knowledge Discovery.

[7]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[8]  Laetitia Vermeulen-Jourdan,et al.  Hybridising rule induction and multi-objective evolutionary search for optimising water distribution systems , 2004, Fourth International Conference on Hybrid Intelligent Systems (HIS'04).

[9]  Ryszard S. Michalski,et al.  The LEM3 implementation of learnable evolution model and its testing on complex function optimization problems , 2006, GECCO.

[10]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[11]  David W. Corne,et al.  The simplest evolution/learning hybrid: LEM with KNN , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[12]  Eugene S. Edgington,et al.  Randomization Tests , 2011, International Encyclopedia of Statistical Science.

[13]  Gilbert Syswerda,et al.  Uniform Crossover in Genetic Algorithms , 1989, ICGA.

[14]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[15]  E. Edgington,et al.  Randomization Tests (3rd ed.) , 1998 .

[16]  Ryszard S. Michalski,et al.  LEARNABLE EVOLUTION MODEL: Evolutionary Processes Guided by Machine Learning , 2004, Machine Learning.

[17]  Pedro Larrañaga,et al.  GA-EDA: hybrid evolutionary algorithm using genetic and estimation of distribution algorithms , 2004 .

[18]  Aiko M. Hormann,et al.  Programs for Machine Learning. Part I , 1962, Inf. Control..

[19]  D. E. Goldberg,et al.  Optimization and Machine Learning , 2022 .

[20]  Pedro Larrañaga,et al.  Towards a New Evolutionary Computation - Advances in the Estimation of Distribution Algorithms , 2006, Towards a New Evolutionary Computation.

[21]  Kenneth A. Kaufman,et al.  Learning from Inconsistent and Noisy Data: The AQ18 Approach , 1999, ISMIS.

[22]  Qingfu Zhang,et al.  Estimation of Distribution Algorithm with 2-opt Local Search for the Quadratic Assignment Problem , 2006, Towards a New Evolutionary Computation.

[23]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[24]  Kenneth A. Kaufman,et al.  Inductive Learning System AQ15c: The Method and User's Guide , 1995 .

[25]  Ryszard S. Michalski,et al.  The LEM3 System for Non-Darwinian Evolutionary Computation and Its Application to Complex Function Optimization , 2005 .

[26]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[27]  Pedro Larrañaga,et al.  Estimation of Distribution Algorithms , 2002, Genetic Algorithms and Evolutionary Computation.

[28]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.