Exploring Hyper-heuristic Methodologies with Genetic Programming

Hyper-heuristics represent a novel search methodology that is motivated by the goal of automating the process of selecting or combining simpler heuristics in order to solve hard computational search problems. An extension of the original hyper-heuristic idea is to generate new heuristics which are not currently known. These approaches operate on a search space of heuristics rather than directly on a search space of solutions to the underlying problem which is the case with most meta-heuristics implementations. In the majority of hyper-heuristic studies so far, a framework is provided with a set of human designed heuristics, taken from the literature, and with good measures of performance in practice. A less well studied approach aims to generate new heuristics from a set of potential heuristic components. The purpose of this chapter is to discuss this class of hyper-heuristics, in which Genetic Programming is the most widely used methodology. A detailed discussion is presented including the steps needed to apply this technique, some representative case studies, a literature review of related work, and a discussion of relevant issues. Our aim is to convey the exciting potential of this innovative approach for automating the heuristic design process.

[1]  Daniele Vigo,et al.  Bin Packing Approximation Algorithms: Combinatorial Analysis , 1999, Handbook of Combinatorial Optimization.

[2]  Jeffrey D. Ullman,et al.  L worst-case performance bounds for rumple one-dimensional packing algorithms siam j , 1974 .

[3]  Peter Nordin,et al.  Genetic programming - An Introduction: On the Automatic Evolution of Computer Programs and Its Applications , 1998 .

[4]  Graham Kendall,et al.  A Hyperheuristic Approach to Scheduling a Sales Summit , 2000, PATAT.

[5]  John R. Koza,et al.  Genetic Programming II , 1992 .

[6]  Steven S. Seiden,et al.  On the online bin packing problem , 2001, JACM.

[7]  Peter Ross,et al.  Learning a Procedure That Can Solve Hard Bin-Packing Problems: A New GA-Based Approach to Hyper-heuristics , 2003, GECCO.

[8]  Rolf Drechsler,et al.  Heuristic Learning Based on Genetic Programming , 2001, EuroGP.

[9]  P. Pardalos,et al.  Handbook of Combinatorial Optimization , 1998 .

[10]  Alex S. Fukunaga,et al.  Automated discovery of composite SAT variable-selection heuristics , 2002, AAAI/IAAI.

[11]  Hishammuddin Asmuni,et al.  A Novel Fuzzy Approach to Evaluate the Quality of Examination Timetabling , 2006, PATAT.

[12]  Peter Ross,et al.  A Promising Hybrid GA/Heuristic Approach for Open-Shop Scheduling Problems , 1994, ECAI.

[13]  Kevin Kok Wai Wong,et al.  Classification of adaptive memetic algorithms: a comparative study , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[14]  Sebastian Thrun,et al.  Learning to Learn: Introduction and Overview , 1998, Learning to Learn.

[15]  Riccardo Poli,et al.  Generating SAT Local-Search Heuristics Using a GP Hyper-Heuristic Framework , 2007, Artificial Evolution.

[16]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[17]  Alexander Nareyek,et al.  Choosing search heuristics by non-stationary reinforcement learning , 2004 .

[18]  Wansoo T. Rhee,et al.  On Line Bin Packing with Items of Random Size , 1993, Math. Oper. Res..

[19]  Vidroha Debroy,et al.  Genetic Programming , 1998, Lecture Notes in Computer Science.

[20]  Graham Kendall,et al.  A Tabu-Search Hyperheuristic for Timetabling and Rostering , 2003, J. Heuristics.

[21]  Thomas Bäck,et al.  An Overview of Parameter Control Methods by Self-Adaption in Evolutionary Algorithms , 1998, Fundam. Informaticae.

[22]  John R. Koza,et al.  Genetic programming 2 - automatic discovery of reusable programs , 1994, Complex Adaptive Systems.

[23]  Natalio Krasnogor,et al.  A Study on the use of ``self-generation'' in memetic algorithms , 2004, Natural Computing.

[24]  Jim E. Smith,et al.  Coevolving Memetic Algorithms: A Review and Progress Report , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[25]  Alex S. Fukunaga,et al.  Automated Discovery of Local Search Heuristics for Satisfiability Testing , 2008, Evolutionary Computation.

[26]  Graham Kendall,et al.  Evolving Bin Packing Heuristics with Genetic Programming , 2006, PPSN.

[27]  Graham Kendall,et al.  Hyper-Heuristics: An Emerging Direction in Modern Search Technology , 2003, Handbook of Metaheuristics.

[28]  Kevin Leyton-Brown,et al.  Performance Prediction and Automated Tuning of Randomized and Parametric Algorithms , 2006, CP.

[29]  Natalio Krasnogor,et al.  Emergence of profitable search strategies based on a simple inheritance mechanism , 2001 .

[30]  Edmund K. Burke,et al.  A simulated annealing based hyperheuristic for determining shipper sizes for storage and transportation , 2007, Eur. J. Oper. Res..

[31]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[32]  John R. Koza,et al.  Genetic Programming III: Darwinian Invention & Problem Solving , 1999 .

[33]  Hugo Terashima-Marín,et al.  Hyper-heuristics and classifier systems for solving 2D-regular cutting stock problems , 2005, GECCO '05.

[34]  Sanja Petrovic,et al.  Case-based heuristic selection for timetabling problems , 2006, J. Sched..

[35]  Zbigniew Michalewicz,et al.  Parameter Control in Evolutionary Algorithms , 2007, Parameter Setting in Evolutionary Algorithms.

[36]  R. Drechsler,et al.  Learning heuristics by genetic algorithms , 1995, Proceedings of ASP-DAC'95/CHDL'95/VLSI'95 with EDA Technofair.

[37]  Benjamin W. Wah,et al.  Genetics-Based Learning of New Heuristics: Rational Scheduling of Experiments and Generalization , 1995, IEEE Trans. Knowl. Data Eng..

[38]  Mauro Birattari,et al.  Tuning Metaheuristics - A Machine Learning Perspective , 2009, Studies in Computational Intelligence.

[39]  Una-May O'Reilly,et al.  Genetic Programming Applied to Compiler Heuristic Optimization , 2003, EuroGP.

[40]  Andy J. Keane,et al.  Meta-Lamarckian learning in memetic algorithms , 2004, IEEE Transactions on Evolutionary Computation.

[42]  Riccardo Poli,et al.  A Field Guide to Genetic Programming , 2008 .

[43]  Alain Delchambre,et al.  A genetic algorithm for bin packing and line balancing , 1992, Proceedings 1992 IEEE International Conference on Robotics and Automation.

[44]  Thomas Stützle,et al.  Stochastic Local Search: Foundations & Applications , 2004 .

[45]  Ender Özcan,et al.  A comprehensive analysis of hyper-heuristics , 2008, Intell. Data Anal..

[46]  Peter Ross,et al.  Solving a Real-World Problem Using an Evolving Heuristically Driven Schedule Builder , 1998, Evolutionary Computation.

[47]  David Pisinger,et al.  A unified heuristic for a large class of Vehicle Routing Problems with Backhauls , 2006, Eur. J. Oper. Res..

[48]  Jonas Mockus,et al.  Application of Bayesian approach to numerical methods of global and stochastic optimization , 1994, J. Glob. Optim..

[49]  Jürgen Schmidhuber,et al.  Learning dynamic algorithm portfolios , 2006, Annals of Mathematics and Artificial Intelligence.

[50]  Sebastian Thrun,et al.  Learning to Learn , 1998, Springer US.

[51]  John R. Koza,et al.  Genetic Programming IV: Routine Human-Competitive Machine Intelligence , 2003 .

[52]  P. Nordin.  Genetic Programming III - Darwinian Invention and Problem Solving , 1999 .

[53]  Wilfried Jakob,et al.  Towards an Adaptive Multimeme Algorithm for Parameter Optimisation Suiting the Engineers' Needs , 2006, PPSN.

[54]  Graham Kendall,et al.  An Investigation of Automated Planograms Using a Simulated Annealing Based Hyper-Heuristic , 2005 .

[55]  Marko Privosnik.  The scalability of evolved on line bin packing heuristics , 2007, 2007 IEEE Congress on Evolutionary Computation.

[56]  Thomas Stützle,et al.  Automatic Algorithm Configuration Based on Local Search , 2007, AAAI.

[57]  Reha Uzsoy,et al.  Rapid Modeling and Discovery of Priority Dispatching Rules: An Autonomous Learning Approach , 2006, J. Sched..

[58]  Benjamin W. Wah,et al.  Teacher: A Genetics Based System for Learning and Generalizing Heuristics , 2000, Soft Computing in Case Based Reasoning.

[59]  Riccardo Poli,et al.  Linear genetic programming of parsimonious metaheuristics , 2007, 2007 IEEE Congress on Evolutionary Computation.

[60]  Steven S. Seiden.  On the Online Bin Packing Problem , 2001, ICALP.

[61]  H. Terashima-Marín,et al.  Evolution of Constraint Satisfaction strategies in examination timetabling , 1999 .

[62]  Sanja Petrovic,et al.  A graph-based hyper-heuristic for educational timetabling problems , 2007, Eur. J. Oper. Res..

[63]  Wilfried Jakob.  HyGLEAM - An Approach to Generally Applicable Hybridization of Evolutionary Algorithms , 2002, PPSN.

[64]  Tad Hogg,et al.  An Economics Approach to Hard Computational Problems , 1997, Science.

[65]  Graham Kendall,et al.  Automatic heuristic generation with genetic programming: evolving a jack-of-all-trades or a master of one , 2007, GECCO '07.