Tackling Large-Scale and Combinatorial Bi-Level Problems With a Genetic Programming Hyper-Heuristic

Combinatorial bi-level optimization remains a challenging topic, especially when the lower-level is an ${\mathcal {NP}}$ -hard problem. In this paper, we tackle large-scale and combinatorial bi-level problems using GP hyper-heuristics, i.e., an approach that permits to train heuristics like a machine learning model. Our contribution aims at targeting the intensive and complex lower-level optimizations that occur when solving a large-scale and combinatorial bi-level problem. For this purpose, we consider hyper-heuristics through heuristic generation. Using a GP hyper-heuristic approach, we train greedy heuristics in order to make them more reliable when encountering unseen lower-level instances that could be generated during bi-level optimization. To validate our approach referred to as GA+AGH, we tackle instances from the bi-level cloud pricing optimization problem (BCPOP) that model the trading interactions between a cloud service provider and cloud service customers. Numerical results demonstrate the abilities of the trained heuristics to cope with the inherent nested structure that makes bi-level optimization problems so hard. Furthermore, it has been shown that training heuristics for lower-level optimization permits to outperform human-based heuristics and metaheuristics which constitute an excellent outcome for bi-level optimization.

[1]  Pascal Bouvry,et al.  Management of an academic HPC cluster: The UL experience , 2014, 2014 International Conference on High Performance Computing & Simulation (HPCS).

[2]  Jerome Bracken,et al.  Mathematical Programs with Optimization Problems in the Constraints , 1973, Oper. Res..

[3]  José-Fernando Camacho-Vallejo,et al.  Analyzing the Performance of a Hybrid Heuristic for Solving a Bilevel Location Problem under Different Approaches to Tackle the Lower Level , 2016 .

[4]  Ender Özcan,et al.  Mapping the performance of heuristics for Constraint Satisfaction , 2010, IEEE Congress on Evolutionary Computation.

[5]  Arnaud Liefooghe,et al.  CoBRA: A cooperative coevolutionary algorithm for bi-level optimization , 2012, 2012 IEEE Congress on Evolutionary Computation.

[6]  Charles E. Blair,et al.  Computational Difficulties of Bilevel Linear Programming , 1990, Oper. Res..

[7]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[8]  Mihai Oltean,et al.  Evolving Evolutionary Algorithms Using Linear Genetic Programming , 2005, Evolutionary Computation.

[9]  Qin Jin,et al.  Bi-level Simulated Annealing Algorithm for Facility Location Problem , 2008, 2008 International Conference on Information Management, Innovation Management and Industrial Engineering.

[10]  Graham Kendall,et al.  Automatic Design of a Hyper-Heuristic Framework With Gene Expression Programming for Combinatorial Optimization Problems , 2015, IEEE Transactions on Evolutionary Computation.

[11]  Xiangyong Li,et al.  A Hierarchical Particle Swarm Optimization for Solving Bilevel Programming Problems , 2006, ICAISC.

[12]  J. Bard Some properties of the bilevel programming problem , 1991 .

[13]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[14]  Martine Labbé,et al.  A Bilevel Model for Toll Optimization on a Multicommodity Transportation Network , 2000, Transp. Sci..

[15]  Kalyanmoy Deb,et al.  An improved bilevel evolutionary algorithm based on Quadratic Approximations , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[16]  Pascal Bouvry,et al.  Bayesian optimization approach of general bi-level problems , 2017, GECCO.

[17]  Dimitris Bertsimas,et al.  Rounding algorithms for covering problems , 1998, Math. Program..

[18]  Ender Özcan,et al.  CHAMP: Creating heuristics via many parameters for online bin packing , 2016, Expert Syst. Appl..

[19]  Athanasios Migdalas,et al.  Bilevel programming in traffic planning: Models, methods and challenge , 1995, J. Glob. Optim..

[20]  Jonathan F. Bard,et al.  A Branch and Bound Algorithm for the Bilevel Programming Problem , 1990, SIAM J. Sci. Comput..

[21]  Graham Kendall,et al.  Grammatical Evolution of Local Search Heuristics , 2012, IEEE Transactions on Evolutionary Computation.

[22]  John E. Beasley,et al.  OR-Library: Distributing Test Problems by Electronic Mail , 1990 .

[23]  Wenjie Zuo,et al.  Bi-level optimization for the cross-sectional shape of a thin-walled car body frame with static stiffness and dynamic frequency stiffness constraints , 2015 .

[24]  Wayne F. Bialas,et al.  On two-level optimization , 1982 .

[25]  Hugo Terashima-Marín,et al.  Grammar-based Genetic Programming for evolving variable ordering heuristics , 2013, 2013 IEEE Congress on Evolutionary Computation.

[26]  José-Fernando Camacho-Vallejo,et al.  A Genetic Algorithm for the Bi-Level Topological Design of Local Area Networks , 2015, PloS one.

[27]  Hugo Terashima-Marín,et al.  Problem-state representations in a hyper-heuristic approach for the 2D irregular BPP , 2010, GECCO '10.

[28]  Graham Kendall,et al.  A Hyperheuristic Approach to Scheduling a Sales Summit , 2000, PATAT.

[29]  S. Minner,et al.  Benders Decomposition for Discrete–Continuous Linear Bilevel Problems with application to traffic network design , 2014 .

[30]  Deniz Aksen,et al.  A bilevel fixed charge location model for facilities under imminent attack , 2012, Comput. Oper. Res..

[31]  El-Ghazali Talbi,et al.  A Taxonomy of Metaheuristics for Bi-level Optimization , 2013 .

[32]  Graham Kendall,et al.  Heuristic, meta-heuristic and hyper-heuristic approaches for fresh produce inventory control and shelf space allocation , 2008, J. Oper. Res. Soc..

[33]  Andrew Koh Solving transportation bi-level programs with Differential Evolution , 2007, 2007 IEEE Congress on Evolutionary Computation.

[34]  Graham Kendall,et al.  A Genetic Programming Hyper-Heuristic Approach for Evolving 2-D Strip Packing Heuristics , 2010, IEEE Transactions on Evolutionary Computation.

[35]  Pascal Bouvry,et al.  A Competitive Approach for Bi-Level Co-Evolution , 2018, 2018 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW).

[36]  Gilles Savard,et al.  The steepest descent direction for the nonlinear bilevel programming problem , 1990, Oper. Res. Lett..

[37]  Marc Parizeau,et al.  DEAP: evolutionary algorithms made easy , 2012, J. Mach. Learn. Res..

[38]  Graham Kendall,et al.  Scheduling English football fixtures over holiday periods , 2008, J. Oper. Res. Soc..

[39]  Ender Özcan,et al.  A genetic programming hyper-heuristic for the multidimensional knapsack problem , 2014, Kybernetes.

[40]  P. Marcotte,et al.  A bilevel model of taxation and its application to optimal highway pricing , 1996 .

[41]  Michel Gendreau,et al.  Hyper-heuristics: a survey of the state of the art , 2013, J. Oper. Res. Soc..

[42]  Graham Kendall,et al.  Automatic heuristic generation with genetic programming: evolving a jack-of-all-trades or a master of one , 2007, GECCO '07.

[43]  Gerald G. Brown,et al.  Interdicting a Nuclear-Weapons Project , 2009, Oper. Res..

[44]  P. Mell,et al.  The NIST Definition of Cloud Computing , 2011 .

[45]  L. N. Vicente,et al.  Multicriteria Approach to Bilevel Optimization , 2006 .

[46]  Mengjie Zhang,et al.  Automated Design of Production Scheduling Heuristics: A Review , 2016, IEEE Transactions on Evolutionary Computation.

[47]  Alex S. Fukunaga,et al.  Evolving Local Search Heuristics for SAT Using Genetic Programming , 2004, GECCO.

[48]  Abdollah Homaifar,et al.  A Bilevel Parameter Tuning Strategy of Partially Connected ANNs , 2015, 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA).

[49]  Graham Kendall,et al.  Grammatical Evolution Hyper-Heuristic for Combinatorial Optimization Problems , 2013, IEEE Transactions on Evolutionary Computation.

[50]  Heinrich von Stackelberg,et al.  Stackelberg (Heinrich von) - The Theory of the Market Economy, translated from the German and with an introduction by Alan T. PEACOCK. , 1953 .

[51]  Graham Kendall,et al.  A simulated annealing hyper-heuristic methodology for flexible decision support , 2012, 4OR.

[52]  Jonathan F. Bard,et al.  Practical Bilevel Optimization: Algorithms and Applications , 1998 .

[53]  Rajkumar Roy,et al.  Bi-level optimisation using genetic algorithm , 2002, Proceedings 2002 IEEE International Conference on Artificial Intelligence Systems (ICAIS 2002).

[54]  Saïd Salhi,et al.  Hyper-heuristic approaches for the response time variability problem , 2011, Eur. J. Oper. Res..

[55]  Michael Florian,et al.  Optimizing frequencies in a transit network: a nonlinear bi-level programming approach , 1995 .

[56]  Matthias Fuchs,et al.  High Performance ATP Systems by Combining Several AI Methods , 1997, IJCAI.

[57]  Gabriela Ochoa,et al.  Grammar-based generation of variable-selection heuristics for constraint satisfaction problems , 2016, Genetic Programming and Evolvable Machines.

[58]  Jerome Bracken,et al.  Defense Applications of Mathematical Programs with Optimization Problems in the Constraints , 1974, Oper. Res..

[59]  Jonathan F. Bard,et al.  An explicit solution to the multi-level programming problem , 1982, Comput. Oper. Res..

[60]  M. Queyranne,et al.  Parametric Integer Programming Algorithm for Bilevel Mixed Integer Programs , 2009, 0907.1298.

[61]  María Cristina Riff,et al.  DVRP: a hard dynamic combinatorial optimisation problem tackled by an evolutionary hyper-heuristic , 2010, J. Heuristics.

[62]  Tom Holvoet,et al.  Optimizing agents with genetic programming: an evaluation of hyper-heuristics in dynamic real-time logistics , 2018, Genetic Programming and Evolvable Machines.

[63]  Nanjangud C. Narendra,et al.  Cloud Pricing Models: A Survey and Position Paper. , 2013, 2013 IEEE International Conference on Cloud Computing in Emerging Markets (CCEM).

[64]  A. Ciric,et al.  A dual temperature simulated annealing approach for solving bilevel programming problems , 1998 .

[65]  Mohit Singh,et al.  LP-Based Algorithms for Capacitated Facility Location , 2014, 2014 IEEE 55th Annual Symposium on Foundations of Computer Science.