Learn to Design the Heuristics for Vehicle Routing Problem

This paper presents an approach to learn the local-search heuristics that iteratively improves the solution of Vehicle Routing Problem (VRP). A local-search heuristics is composed of a destroy operator that destructs a candidate solution, and a following repair operator that rebuilds the destructed one into a new one. The proposed neural network, as trained through actor-critic framework, consists of an encoder in form of a modified version of Graph Attention Network where node embeddings and edge embeddings are integrated, and a GRU-based decoder rendering a pair of destroy and repair operators. Experiment results show that it outperforms both the traditional heuristics algorithms and the existing neural combinatorial optimization for VRP on medium-scale data set, and is able to tackle the large-scale data set (e.g., over 400 nodes) which is a considerable challenge in this area. Moreover, the need for expertise and handcrafted heuristics design is eliminated due to the fact that the proposed network learns to design the heuristics with a better performance. Our implementation is available online.

[1]  Yoshua Bengio,et al.  Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizon , 2018, Eur. J. Oper. Res..

[2]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[3]  Teodor Gabriel Crainic,et al.  An adaptive large neighborhood search heuristic for Two-Echelon Vehicle Routing Problems arising in city logistics , 2012, Comput. Oper. Res..

[4]  Paul Shaw,et al.  Using Constraint Programming and Local Search Methods to Solve Vehicle Routing Problems , 1998, CP.

[5]  Joshua B. Tenenbaum,et al.  Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation , 2016, NIPS.

[6]  Samy Bengio,et al.  Neural Combinatorial Optimization with Reinforcement Learning , 2016, ICLR.

[7]  David Pisinger,et al.  An Adaptive Large Neighborhood Search Heuristic for the Pickup and Delivery Problem with Time Windows , 2006, Transp. Sci..

[8]  Mwp Martin Savelsbergh,et al.  A parallel insertion heuristic for vehicle routing with side constraints , 1990 .

[9]  Bruce L. Golden,et al.  A library of local search heuristics for the vehicle routing problem , 2010, Math. Program. Comput..

[10]  Navdeep Jaitly,et al.  Pointer Networks , 2015, NIPS.

[11]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[12]  Yishay Mansour,et al.  Policy Gradient Methods for Reinforcement Learning with Function Approximation , 1999, NIPS.

[13]  Gilbert Laporte,et al.  Tabu Search Heuristics for the Vehicle Routing Problem , 2005 .

[14]  Yuandong Tian,et al.  Learning to Perform Local Rewriting for Combinatorial Optimization , 2019, NeurIPS.

[15]  Qiang Ma,et al.  Combinatorial Optimization by Graph Pointer Networks and Hierarchical Reinforcement Learning , 2019, ArXiv.

[16]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[17]  Michel Gendreau,et al.  An efficient variable neighborhood search heuristic for very large scale vehicle routing problems , 2007, Comput. Oper. Res..

[18]  Lawrence V. Snyder,et al.  Reinforcement Learning for Solving the Vehicle Routing Problem , 2018, NeurIPS.

[19]  Ruhan He,et al.  Combining Nearest Neighbor Search with Tabu Search for Large-Scale Vehicle Routing Problem , 2012 .

[20]  Max Welling,et al.  Buy 4 REINFORCE Samples, Get a Baseline for Free! , 2019, DeepRLStructPred@ICLR.

[21]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[22]  Max Welling,et al.  Attention, Learn to Solve Routing Problems! , 2018, ICLR.

[23]  Greet Van den Berghe,et al.  Slack Induction by String Removals for Vehicle Routing Problems , 2020, Transp. Sci..