A selection hyperheuristic guided by Thompson sampling for numerical optimization

Selection hyper-heuristics have been increasingly and successfully applied to numerical and discrete optimization problems. This paper proposes HHTS, a hyper-heuristic (HH) based on the Thompson Sampling (TS) mechanism to select combinations of low-level heuristics aiming to provide solutions for various continuous single-objective optimization benchmarks. Thompson Sampling is modeled in the present paper as a Beta Bernoulli sampler considering the increase/decrease of diversity among population individuals to measure the success/failure during the search. In the experiments, HHTS (a generic evolutionary algorithm generated by TS) is compared with five well-known evolutionary algorithms. Results indicate that, despite requiring less computational effort, HHTS's performance is similar or better than the other algorithm for most instances and in 50% of the cases it is capable of achieving the global optimum.

[1]  Liang Tang,et al.  Automatic ad format selection via contextual bandits , 2013, CIKM.

[2]  Lihong Li,et al.  An Empirical Evaluation of Thompson Sampling , 2011, NIPS.

[3]  C. Borror.  Practical Nonparametric Statistics, 3rd Ed. , 2001 .

[4]  Dalila Boughaci,et al.  A synergy Thompson sampling hyper‐heuristic for the feature selection problem , 2020, Comput. Intell..

[5]  Dirk Thierens,et al.  Convergence Models of Genetic Algorithm Selection Schemes , 1994, PPSN.

[6]  Yuri Malitsky,et al.  Model-Based Genetic Algorithms for Algorithm Configuration , 2015, IJCAI.

[7]  Shipra Agrawal,et al.  Analysis of Thompson Sampling for the Multi-armed Bandit Problem , 2011, COLT.

[8]  W. R. Thompson.  ON THE LIKELIHOOD THAT ONE UNKNOWN PROBABILITY EXCEEDS ANOTHER IN VIEW OF THE EVIDENCE OF TWO SAMPLES , 1933 .

[9]  Nasser R. Sabar,et al.  Hyper-heuristic Based Local Search for Combinatorial Optimisation Problems , 2018, Australasian Conference on Artificial Intelligence.

[10]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[11]  Anne Auger,et al.  Tutorial CMA-ES: evolution strategies and covariance matrix adaptation , 2012, Annual Conference on Genetic and Evolutionary Computation.

[12]  Miao Li,et al.  A Hyperheuristic Approach for Intercell Scheduling With Single Processing Machines and Batch Processing Machines , 2015, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[13]  T. L. Lai Andherbertrobbins.  Asymptotically Efficient Adaptive Allocation Rules , 2022 .

[14]  Kevin Leyton-Brown,et al.  SATzilla: Portfolio-based Algorithm Selection for SAT , 2008, J. Artif. Intell. Res..

[15]  Edmund K. Burke,et al.  Recent advances in selection hyper-heuristics , 2020, Eur. J. Oper. Res..

[16]  Sanja Petrovic,et al.  HyFlex: A Benchmark Framework for Cross-Domain Heuristic Search , 2011, EvoCOP.

[17]  Adam Lipowski,et al.  Roulette-wheel selection via stochastic acceptance , 2011, ArXiv.

[18]  Pinar Civicioglu,et al.  Transforming geocentric cartesian coordinates to geodetic coordinates by using differential search algorithm , 2012, Comput. Geosci..

[19]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[20]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[21]  Thomas Bartz-Beielstein,et al.  SPOT: A Toolbox for Interactive and Automatic Tuning in the R Environment , 2010 .

[22]  Mark Hoogendoorn,et al.  Parameter Control in Evolutionary Algorithms: Trends and Challenges , 2015, IEEE Transactions on Evolutionary Computation.

[23]  Patrick Siarry,et al.  A survey on optimization metaheuristics , 2013, Inf. Sci..

[24]  Djallel Bouneffouf,et al.  A Survey on Practical Applications of Multi-Armed and Contextual Bandits , 2019, ArXiv.

[25]  Andries Petrus Engelbrecht,et al.  Analysis of selection hyper-heuristics for population-based meta-heuristics in real-valued dynamic optimization , 2018, Swarm Evol. Comput..

[26]  Nenghai Yu,et al.  Thompson Sampling for Budgeted Multi-Armed Bandits , 2015, IJCAI.

[27]  Riccardo Poli,et al.  Schema Theory for Genetic Programming with One-Point Crossover and Point Mutation , 1997, Evolutionary Computation.

[28]  K. Deb,et al.  Real-coded evolutionary algorithms with parent-centric recombination , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[29]  Graham Kendall,et al.  A Genetic Programming Hyper-Heuristic Approach for Evolving 2-D Strip Packing Heuristics , 2010, IEEE Transactions on Evolutionary Computation.

[30]  Carlos A. Coello Coello,et al.  On the use of particle swarm optimization with multimodal functions , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[31]  Dan Boneh,et al.  On genetic algorithms , 1995, COLT '95.

[32]  Aurélien Garivier,et al.  On Bayesian Upper Confidence Bounds for Bandit Problems , 2012, AISTATS.

[33]  Dalila Boughaci,et al.  A multilevel synergy Thompson sampling hyper-heuristic for solving Max-SAT , 2019, Intell. Decis. Technol..

[34]  H. Robbins.  Some aspects of the sequential design of experiments , 1952 .

[35]  Benjamin Van Roy,et al.  A Tutorial on Thompson Sampling , 2017, Found. Trends Mach. Learn..

[36]  Steven L. Scott,et al.  A modern Bayesian look at the multi-armed bandit , 2010 .

[37]  Aurora Trinidad Ramirez Pozo,et al.  A Multi-Armed Bandit selection strategy for Hyper-heuristics , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[38]  Eduardo F. Morales,et al.  An Introduction to Reinforcement Learning , 2011 .

[39]  Leslie Pérez Cáceres,et al.  The irace package: Iterated racing for automatic algorithm configuration , 2016 .

[40]  Fawaz Alanazi.  Adaptive Thompson Sampling for hyper-heuristics , 2016, 2016 IEEE Symposium Series on Computational Intelligence (SSCI).

[41]  Ole-Christoffer Granmo,et al.  Solving two-armed Bernoulli bandit problems using a Bayesian learning automaton , 2010, Int. J. Intell. Comput. Cybern..

[42]  Girish Chowdhary,et al.  The Explore–Exploit Dilemma in Nonstationary Decision Making under Uncertainty , 2015 .

[43]  Robert Sabourin,et al.  Review and Study of Genotypic Diversity Measures for Real-Coded Representations , 2012, IEEE Transactions on Evolutionary Computation.

[44]  Andrea Roli,et al.  MAGMA: a multiagent architecture for metaheuristics , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[45]  Diego Oliva,et al.  A Bayesian based Hyper-Heuristic approach for global optimization , 2019, 2019 IEEE Congress on Evolutionary Computation (CEC).

[46]  Carolina P. de Almeida,et al.  A New Hyper-Heuristic Based on a Contextual Multi-Armed Bandit for Many-Objective Optimization , 2018, 2018 IEEE Congress on Evolutionary Computation (CEC).

[47]  Hamid R. Tizhoosh,et al.  Opposition-Based Learning: A New Scheme for Machine Intelligence , 2005, International Conference on Computational Intelligence for Modelling, Control and Automation and International Conference on Intelligent Agents, Web Technologies and Internet Commerce (CIMCA-IAWTIC'06).