A hybrid multi-population framework for dynamic environments combining online and offline learning

Population based incremental learning algorithms and selection hyper-heuristics are highly adaptive methods which can handle different types of dynamism that may occur while a given problem is being solved. In this study, we present an approach based on a multi-population framework hybridizing these methods to solve dynamic environment problems. A key feature of the hybrid approach is the utilization of offline and online learning methods at successive stages. The performance of our approach along with the influence of different heuristic selection methods used within the selection hyper-heuristic is investigated over a range of dynamic environments produced by a well known benchmark generator as well as a real world problem, referred to as the Unit Commitment Problem. The empirical results show that the proposed approach using a particular hyper-heuristic outperforms some of the best known approaches in literature on the dynamic environment problems dealt with.

[1]  John R. Woodward,et al.  Hyper-Heuristics , 2015, GECCO.

[2]  Mark Wineberg,et al.  Enhancing the GA's Ability to Cope with Dynamic Environments , 2000, GECCO.

[3]  Maria E. Orlowska,et al.  Extending a class of continuous estimation of distribution algorithms to dynamic problems , 2008, Optim. Lett..

[4]  Gilbert Owusu,et al.  An application of EDA and GA to dynamic pricing , 2007, GECCO '07.

[5]  Anabela Simões,et al.  Improving prediction in evolutionary algorithms for dynamic environments , 2009, GECCO.

[6]  Michel Gendreau,et al.  Hyper-heuristics: a survey of the state of the art , 2013, J. Oper. Res. Soc..

[7]  Shengxiang Yang,et al.  Constructing dynamic test environments for genetic algorithms based on problem difficulty , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[8]  Belgin Emre Turkay,et al.  A novel differential evolution application to short-term electrical power generation scheduling , 2011 .

[9]  Belgin Emre Turkay,et al.  Evolutionary Algorithms for the Unit Commitment Problem , 2008 .

[10]  Xin Yao,et al.  Population-Based Incremental Learning With Associative Memory for Dynamic Environments , 2008, IEEE Transactions on Evolutionary Computation.

[11]  Fernando José Von Zuben,et al.  Online learning in estimation of distribution algorithms for dynamic environments , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[12]  Naim Dahnoun,et al.  Studies in Computational Intelligence , 2013 .

[13]  Peter I. Cowling,et al.  Hyperheuristics: Recent Developments , 2008, Adaptive and Multilevel Metaheuristics.

[14]  Yuping Wang,et al.  Multi-population Based Univariate Marginal Distribution Algorithm for Dynamic Optimization Problems , 2010, J. Intell. Robotic Syst..

[15]  Yuping Wang,et al.  Multi-population and diffusion UMDA for dynamic multimodal problems , 2010 .

[16]  Hui Cheng,et al.  Multi-population Genetic Algorithms with Immigrants Scheme for Dynamic Shortest Path Routing Problems in Mobile Ad Hoc Networks , 2010, EvoApplications.

[17]  Anastasios G. Bakirtzis,et al.  Genetic algorithm solution to the economic dispatch problem , 1994 .

[18]  A. Sima Etaner-Uyar,et al.  An Investigation of Selection Hyper-heuristics in Dynamic Environments , 2011, EvoApplications.

[19]  Jiri Ocenasek,et al.  Bayesian Optimization Algorithms for Dynamic Problems , 2006, EvoWorkshops.

[20]  Fred Glover,et al.  PROBABILISTIC AND PARAMETRIC LEARNING COMBINATIONS OF LOCAL JOB SHOP SCHEDULING RULES , 1963 .

[21]  A. Sima Etaner-Uyar,et al.  Heuristic selection in a multi-phase hybrid approach for dynamic environments , 2012, 2012 12th UK Workshop on Computational Intelligence (UKCI).

[22]  Demin Xu,et al.  On the effect of environment-triggered population diversity compensation methods for memory enhanced UMDA , 2011, Proceedings of the 30th Chinese Control Conference.

[23]  A. Sima Etaner-Uyar,et al.  Selection hyper-heuristics in dynamic environments , 2013, J. Oper. Res. Soc..

[24]  Jürgen Branke,et al.  A Multi-population Approach to Dynamic Optimization Problems , 2000 .

[25]  G. Sheblé,et al.  Power generation operation and control — 2nd edition , 1996 .

[26]  Ender Özcan,et al.  A comprehensive analysis of hyper-heuristics , 2008, Intell. Data Anal..

[27]  Graham Kendall,et al.  A Hyperheuristic Approach to Scheduling a Sales Summit , 2000, PATAT.

[28]  GhoshAshish,et al.  Univariate marginal distribution algorithms for non-stationary optimization problems , 2004 .

[29]  Shengxiang Yang,et al.  Explicit Memory Schemes for Evolutionary Algorithms in Dynamic Environments , 2007, Evolutionary Computation in Dynamic and Uncertain Environments.

[30]  Allen J. Wood,et al.  Power Generation, Operation, and Control , 1984 .

[31]  Giovanni Iacca,et al.  Ockham's Razor in memetic computing: Three stage optimal memetic exploration , 2012, Inf. Sci..

[32]  Heinz Mühlenbein,et al.  Univariate marginal distribution algorithms for non-stationary optimization problems , 2004, Int. J. Knowl. Based Intell. Eng. Syst..

[33]  Shengxiang Yang,et al.  Evolutionary Computation in Dynamic and Uncertain Environments , 2007, Studies in Computational Intelligence.

[34]  Ronald W. Morrison,et al.  Designing Evolutionary Algorithms for Dynamic Environments , 2004, Natural Computing Series.

[35]  Shumeet Baluja,et al.  A Method for Integrating Genetic Search Based Function Optimization and Competitive Learning , 1994 .

[36]  Pedro Larrañaga,et al.  Estimation of Distribution Algorithms , 2002, Genetic Algorithms and Evolutionary Computation.

[37]  A. Sima Etaner-Uyar,et al.  A Framework to Hybridize PBIL and a Hyper-heuristic for Dynamic Environments , 2012, PPSN.

[38]  Carlos Cotta,et al.  Adaptive and multilevel metaheuristics , 2008 .

[39]  Anabela Simões,et al.  Evolutionary Algorithms for Dynamic Environments: Prediction Using Linear Regression and Markov Chains , 2008, PPSN.

[40]  Kay Chen Tan,et al.  A Multi-Facet Survey on Memetic Computation , 2011, IEEE Transactions on Evolutionary Computation.

[41]  Shengxiang Yang,et al.  Environment identification-based memory scheme for estimation of distribution algorithms in dynamic environments , 2011, Soft Comput..

[42]  Shengxiang Yang,et al.  Evolutionary Computation in Dynamic and Uncertain Environments (Studies in Computational Intelligence) , 2007 .

[43]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[44]  Xin Yao,et al.  Experimental study on population-based incremental learning algorithms for dynamic optimization problems , 2005, Soft Comput..

[45]  Peter A. N. Bosman,et al.  Learning, anticipation and time-deception in evolutionary online dynamic optimization , 2005, GECCO '05.

[46]  Jürgen Branke,et al.  Evolutionary Optimization in Dynamic Environments , 2001, Genetic Algorithms and Evolutionary Computation.

[47]  Emma Hart,et al.  A Comparison of Dominance Mechanisms and Simple Mutation on Non-stationary Problems , 1998, PPSN.

[48]  Anabela Simões,et al.  Prediction in evolutionary algorithms for dynamic environments using markov chains and nonlinear regression , 2009, GECCO.

[49]  María Cristina Riff,et al.  Solving the short-term electrical generation scheduling problem by an adaptive evolutionary approach , 2007, Eur. J. Oper. Res..

[50]  Berna Kiraz,et al.  Hyper-heuristic approaches for the dynamic generalized assignment problem , 2010, 2010 10th International Conference on Intelligent Systems Design and Applications.

[51]  Alexander Nareyek,et al.  Choosing search heuristics by non-stationary reinforcement learning , 2004 .

[52]  Stephen F. Smith,et al.  Using memory models to improve adaptive efficiency in dynamic problems , 2009, 2009 IEEE Symposium on Computational Intelligence in Scheduling.

[53]  Edmund K. Burke,et al.  A Reinforcement Learning - Great-Deluge Hyper-Heuristic for Examination Timetabling , 2010, Int. J. Appl. Metaheuristic Comput..

[54]  Cao Yong,et al.  A novel updating strategy for associative memory scheme in cyclic dynamic environments , 2010, Third International Workshop on Advanced Computational Intelligence.

[55]  A. Sima Etaner-Uyar,et al.  A new population based adaptive domination change mechanism for diploid genetic algorithms in dynamic environments , 2005, Soft Comput..

[56]  Rasmus K. Ursem,et al.  Multinational GAs: Multimodal Optimization Techniques in Dynamic Environments , 2000, GECCO.

[57]  A. Sima Etaner-Uyar,et al.  An Ant-Based Selection Hyper-heuristic for Dynamic Environments , 2013, EvoApplications.

[58]  Agostinho C. Rosa,et al.  UMDAs for dynamic optimization problems , 2008, GECCO '08.

[59]  Carlos Cotta,et al.  Memetic algorithms and memetic computing optimization: A literature review , 2012, Swarm Evol. Comput..

[60]  Alice E. Smith,et al.  A Seeded Memetic Algorithm for Large Unit Commitment Problems , 2002, J. Heuristics.

[61]  Jürgen Branke,et al.  Evolutionary optimization in uncertain environments-a survey , 2005, IEEE Transactions on Evolutionary Computation.

[62]  Graham Kendall,et al.  Search Methodologies: Introductory Tutorials in Optimization and Decision Support Techniques , 2013 .

[63]  Pablo Moscato,et al.  Handbook of Memetic Algorithms , 2011, Studies in Computational Intelligence.

[64]  Carlos Cruz,et al.  Optimization in dynamic environments: a survey on problems, methods and measures , 2011, Soft Comput..

[65]  Shengxiang Yang,et al.  Memory-enhanced univariate marginal distribution algorithms for dynamic optimization problems , 2005, 2005 IEEE Congress on Evolutionary Computation.

[66]  Jürgen Branke,et al.  Memory enhanced evolutionary algorithms for changing optimization problems , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[67]  Shengxiang Yang,et al.  Hyper-learning for population-based incremental learning in dynamic environments , 2009, 2009 IEEE Congress on Evolutionary Computation.

[68]  Shengxiang Yang,et al.  Population-based incremental learning with memory scheme for changing environments , 2005, GECCO '05.

[69]  Shingo Mabu,et al.  Probabilistic model building Genetic Network Programming using multiple probability vectors , 2010, TENCON 2010 - 2010 IEEE Region 10 Conference.