A Permutational Boltzmann Machine with Parallel Tempering for Solving Combinatorial Optimization Problems

Boltzmann Machines are recurrent neural networks that have been used extensively in combinatorial optimization due to their simplicity and ease of parallelization. This paper introduces the Permutational Boltzmann Machine, a neural network capable of solving permutation optimization problems. We implement this network in combination with a Parallel Tempering algorithm with varying degrees of parallelism ranging from a single-thread variant to a multi-threaded system using a 64-core CPU with SIMD instructions. We benchmark the performance of this new system on Quadratic Assignment Problems, using some of the most difficult known instances, and show that our parallel system performs in excess of 100\(\times \) faster than any known dedicated solver, including those implemented on CPU clusters, GPUs, and FPGAs.

[1]  Zvi Drezner,et al.  Recent Advances for the Quadratic Assignment Problem with Special Emphasis on Instances that are Difficult for Meta-Heuristic Methods , 2005, Ann. Oper. Res..

[2]  Amine Agharghor,et al.  Improved Hunting Search Algorithm for the Quadratic Assignment Problem , 2019, Indonesian Journal of Electrical Engineering and Computer Science.

[3]  Michael W Deem,et al.  Parallel tempering: theory, applications, and new perspectives. , 2005, Physical chemistry chemical physics : PCCP.

[4]  Daniel Diaz,et al.  Hybridization as Cooperative Parallelism for the Quadratic Assignment Problem , 2016, HM.

[5]  Kathryn A. Dowsland,et al.  Simulated Annealing , 1989, Encyclopedia of GIS.

[6]  Safak Bayir,et al.  A cooperative GPU-based Parallel Multistart Simulated Annealing algorithm for Quadratic Assignment Problem , 2018, Engineering Science and Technology, an International Journal.

[7]  Daniel Diaz,et al.  Solving the Quadratic Assignment Problem with Cooperative Parallel Extremal Optimization , 2016, EvoCOP.

[8]  Manuel Graña,et al.  Solving Satisfiability Via Boltzmann Machines , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  E. Aarts,et al.  Boltzmann machines for travelling salesman problems , 1989 .

[10]  K. Hukushima,et al.  Exchange Monte Carlo Method and Application to Spin Glass Simulations , 1995, cond-mat/9512035.

[11]  Yagmur Aksan,et al.  A stagnation-aware cooperative parallel breakout local search algorithm for the quadratic assignment problem , 2017, Comput. Ind. Eng..

[12]  Emile H. L. Aarts,et al.  Combinatorial Optimization on a Boltzmann Machine , 1989, J. Parallel Distributed Comput..

[13]  Fred W. Glover,et al.  Quantum Bridge Analytics I: a tutorial on formulating and using QUBO models , 2018, 4OR.

[14]  Daniel Diaz,et al.  Weaving of Metaheuristics with Cooperative Parallelism , 2018, PPSN.

[15]  Gintaras Palubeckis,et al.  An Algorithm for Construction of Test Cases for the Quadratic Assignment Problem , 2000, Informatica.

[16]  Emile H. L. Aarts,et al.  Boltzmann machines as a model for parallel annealing , 1991, Algorithmica.

[17]  Alessandro De Gloria,et al.  Clustered Boltzmann Machines: Massively Parallel Architectures for Constrained Optimization Problems , 1993, Parallel Comput..

[18]  Wang,et al.  Replica Monte Carlo simulation of spin glasses. , 1986, Physical review letters.

[19]  Franz Rendl,et al.  QAPLIB – A Quadratic Assignment Problem Library , 1997, J. Glob. Optim..

[20]  Hirotaka Tamura,et al.  Replica Exchange MCMC Hardware With Automatic Temperature Selection and Parallel Trial , 2020, IEEE Transactions on Parallel and Distributed Systems.

[21]  Shigeyoshi Tsutsui,et al.  ACO on Multiple GPUs with CUDA for Faster Solution of QAPs , 2012, PPSN.

[22]  E. Lawler.  The Quadratic Assignment Problem , 1963 .