An apprenticeship learning hyper-heuristic for vehicle routing in HyFlex

Apprenticeship learning occurs via observations while an expert is in action. A hyper-heuristic is a search method or a learning mechanism that controls a set of low level heuristics or combines different heuristic components to generate heuristics for solving a given computationally hard problem. In this study, we investigate into a novel apprenticeship-learning-based approach which is used to automatically generate a hyper-heuristic for vehicle routing. This approach itself can be considered as a hyper-heuristic which operates in a train and test fashion. A state-of-the-art hyper-heuristic is chosen as an expert which is the winner of a previous hyper-heuristic competition. Trained on small vehicle routing instances, the learning approach yields various classifiers, each capturing different actions that the expert hyper-heuristic performs during the search process. Those classifiers are then used to produce a hyper-heuristic which is potentially capable of generalizing the actions of the expert hyper-heuristic while solving the unseen instances. The experimental results on vehicle routing using the Hyper-heuristic Flexible (HyFlex) framework shows that the apprenticeship-learning-based hyper-heuristic delivers an outstanding performance when compared to the expert and some other previously proposed hyper-heuristics.

[1]  F. Glover,et al.  Handbook of Metaheuristics , 2019, International Series in Operations Research & Management Science.

[2]  Fred Glover,et al.  PROBABILISTIC AND PARAMETRIC LEARNING COMBINATIONS OF LOCAL JOB SHOP SCHEDULING RULES , 1963 .

[3]  Aiko M. Hormann,et al.  Programs for Machine Learning. Part I , 1962, Inf. Control..

[4]  Reha Uzsoy,et al.  Rapid Modeling and Discovery of Priority Dispatching Rules: An Autonomous Learning Approach , 2006, J. Sched..

[5]  Sanja Petrovic,et al.  HyFlex: A Benchmark Framework for Cross-Domain Heuristic Search , 2011, EvoCOP.

[6]  Graham Kendall,et al.  An investigation of a hyperheuristic genetic algorithm applied to a trainer scheduling problem , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[7]  Graham Kendall,et al.  Exploring Hyper-heuristic Methodologies with Genetic Programming , 2009 .

[8]  Paolo Toth,et al.  Models, relaxations and exact approaches for the capacitated vehicle routing problem , 2002, Discret. Appl. Math..

[9]  Michel Gendreau,et al.  Hyper-heuristics: a survey of the state of the art , 2013, J. Oper. Res. Soc..

[10]  Ethem Alpaydin,et al.  Introduction to Machine Learning (Adaptive Computation and Machine Learning) , 2004 .

[11]  Karl Sims,et al.  Artificial evolution for computer graphics , 1991, SIGGRAPH.

[12]  A. Sima Etaner-Uyar,et al.  Generalizing Hyper-heuristics via Apprenticeship Learning , 2013, EvoCOP.

[13]  George B. Dantzig,et al.  The Truck Dispatching Problem , 1959 .

[14]  Michel Gendreau,et al.  A review of dynamic vehicle routing problems , 2013, Eur. J. Oper. Res..

[15]  Rajeev Kumar,et al.  Evolution of hyperheuristics for the biobjective 0/1 knapsack problem by multiobjective genetic programming , 2008, GECCO '08.

[16]  Graham Kendall,et al.  Automatic Design of a Hyper-Heuristic Framework With Gene Expression Programming for Combinatorial Optimization Problems , 2015, IEEE Transactions on Evolutionary Computation.

[17]  Michel Gendreau,et al.  Vehicle Routing and Adaptive Iterated Local Search within the HyFlex Hyper-heuristic Framework , 2012, LION.

[18]  Graham Kendall,et al.  A Genetic Programming Hyper-Heuristic Approach for Evolving 2-D Strip Packing Heuristics , 2010, IEEE Transactions on Evolutionary Computation.

[19]  Hideyuki Takagi,et al.  Interactive evolutionary computation: fusion of the capabilities of EC optimization and human evaluation , 2001, Proc. IEEE.

[20]  Josh C. Bongard,et al.  Combining fitness-based search and user modeling in evolutionary robotics , 2013, GECCO '13.

[21]  Josh C. Bongard,et al.  Avoiding local optima with user demonstrations and low-level control , 2013, 2013 IEEE Congress on Evolutionary Computation.

[22]  Graham Kendall,et al.  A Classification of Hyper-heuristic Approaches , 2010 .

[23]  Paolo Toth,et al.  The Vehicle Routing Problem , 2002, SIAM monographs on discrete mathematics and applications.

[24]  Li-Chen Fu,et al.  A VNS-based hyper-heuristic with adaptive computational budget of local search , 2012, 2012 IEEE Congress on Evolutionary Computation.

[25]  Pieter Abbeel,et al.  Apprenticeship learning via inverse reinforcement learning , 2004, ICML.

[26]  Nicolas Jozefowiez,et al.  The vehicle routing problem: Latest advances and new challenges , 2007 .

[27]  Peter I. Cowling,et al.  Mining the data from a hyperheuristic approach using associative classification , 2008, Expert Syst. Appl..

[28]  Sai Ho Chung,et al.  Survey of Green Vehicle Routing Problem: Past and future trends , 2014, Expert Syst. Appl..

[29]  Ender Özcan,et al.  Generation of VNS Components with Grammatical Evolution for Vehicle Routing , 2013, EuroGP.

[30]  Patrick De Causmaecker,et al.  An Intelligent Hyper-Heuristic Framework for CHeSC 2011 , 2012, LION.

[31]  Chi Fai Cheung,et al.  A Hyper-Heuristic Inspired by Pearl Hunting , 2012, LION.

[32]  Graham Kendall,et al.  A Hyperheuristic Approach to Scheduling a Sales Summit , 2000, PATAT.