Learning, anticipation and time-deception in evolutionary online dynamic optimization

In this paper we focus on an important source of problem-difficulty in (online) dynamic optimization problems that has so far received significantly less attention than the traditional shifting of optima. Intuitively put, decisions taken now (i.e. setting the problem variables to certain values) may influence the score that can be obtained in the future. We indicate how such time-linkage can deceive an optimizer and cause it to find a suboptimal solution trajectory. We then propose a means to address time-linkage: predict the future by learning from the past. We formalize this means in an algorithmic framework. Also, we indicate why evolutionary algorithms are specifically of interest in this framework. We have performed experiments with two new benchmark problems that contain time-linkage. The results show, as a proof of principle, that in the presence of time-linkage EAs based upon this framework can obtain better results than classic EAs that do not predict the future.

[1]  John J. Grefenstette,et al.  Evolvability in dynamic fitness landscapes: a genetic algorithm approach , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[2]  Jürgen Branke,et al.  Evolutionary Optimization in Dynamic Environments , 2001, Genetic Algorithms and Evolutionary Computation.

[3]  Maurice G. Kendall,et al.  The advanced theory of statistics , 1945 .

[4]  Rasmus K. Ursem,et al.  Multinational GAs: Multimodal Optimization Techniques in Dynamic Environments , 2000, GECCO.

[5]  D. G. Beech,et al.  The Advanced Theory of Statistics. Volume 2: Inference and Relationship. , 1962 .

[6]  Michael R. Caputo.  Foundations of Dynamic Economic Analysis , 2005 .

[7]  Dirk Thierens,et al.  Advancing continuous IDEAs with mixture distributions and factorization selection metrics , 2001 .

[8]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[9]  Kenneth A. De Jong,et al.  Evolving in a Changing World , 1999, ISMIS.

[10]  Dirk Thierens,et al.  The Naive MIDEA: A Baseline Multi-objective EA , 2005, EMO.

[11]  Jano I. van Hemert,et al.  Dynamic Routing Problems with Fruitful Regions: Models and Evolutionary Computation , 2004, PPSN.

[12]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[13]  Warren B. Powell,et al.  A COMPARATIVE REVIEW OF ALTERNATIVE ALGORITHMS FOR THE DYNAMIC VEHICLE ALLOCATION PROBLEM , 1988 .

[14]  Joanne H. Walker,et al.  Combining Evolutionary And Non-evolutionary Methods In Tracking Dynamic Global Optima , 2002, GECCO.

[15]  Balaraman Ravindran,et al.  C3: Reinforcement Learning , 1996 .

[16]  Peter J. Angeline,et al.  Tracking Extrema in Dynamic Environments , 1997, Evolutionary Programming.

[17]  Katja Verbeeck,et al.  A “Futurist” approach to dynamic environments , 2000 .

[18]  Richard S. Sutton,et al.  Temporal credit assignment in reinforcement learning , 1984 .

[19]  Russell Beale,et al.  Handbook of Neural Computation , 1996 .

[20]  Hans-Georg Beyer,et al.  Random Dynamics Optimum Tracking with Evolution Strategies , 2002, PPSN.

[21]  Dirk Thierens,et al.  Scalability Problems of Simple Genetic Algorithms , 1999, Evolutionary Computation.

[22]  Jürgen Branke,et al.  A Multi-population Approach to Dynamic Optimization Problems , 2000 .

[23]  Jürgen Branke,et al.  Memory enhanced evolutionary algorithms for changing optimization problems , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[24]  Pedro Larrañaga,et al.  Optimization in Continuous Domains by Learning and Simulation of Gaussian Networks , 2000 .

[25]  Andrew L. Tuson,et al.  Diversity does not necessarily imply adaptability , 2003 .

[26]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[27]  ten Hmm Huub Eikelder,et al.  Finite population models of dynamic optimization with alternating fitness functions , 2003 .

[28]  Jürgen Branke,et al.  Anticipation in Dynamic Optimization: The Scheduling Case , 2000, PPSN.

[29]  Mark Wineberg,et al.  Enhancing the GA's Ability to Cope with Dynamic Environments , 2000, GECCO.

[30]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .