Learning Local Search Heuristics for Boolean Satisfiability

We present an approach to learn SAT solver heuristics from scratch through deep reinforcement learning with a curriculum. In particular, we incorporate a graph neural network in a stochastic local search algorithm to act as the variable selection heuristic. We consider Boolean satisfiability problems from different classes and learn specialized heuristics for each class. Although we do not aim to compete with the state-of-the-art SAT solvers in run time, we demonstrate that the learned heuristics allow us to find satisfying assignments in fewer steps compared to a generic heuristic, and we provide analysis of our results through experiments.

[1]  Krzysztof Czarnecki,et al.  Exponential Recency Weighted Average Branching Heuristic for SAT Solvers , 2016, AAAI.

[2]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[3]  Le Song,et al.  2 Common Formulation for Greedy Algorithms on Graphs , 2018 .

[4]  Stephen A. Cook,et al.  The complexity of theorem-proving procedures , 1971, STOC.

[5]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[6]  Nikolaj Bjørner,et al.  Guiding High-Performance SAT Solvers with Unsat-Core Predictions , 2019, SAT.

[7]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[8]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[9]  Kevin Leyton-Brown,et al.  SATenstein: Automatically Building Local Search SAT Solvers from Components , 2009, IJCAI.

[10]  James M. Crawford,et al.  Experimental Results on the Crossover Point in Random 3-SAT , 1996, Artif. Intell..

[11]  J. Dall,et al.  Random geometric graphs. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[12]  Kevin Leyton-Brown,et al.  SATzilla: Portfolio-based Algorithm Selection for SAT , 2008, J. Artif. Intell. Res..

[13]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[14]  Van H. Vu,et al.  Generating Random Regular Graphs , 2003, STOC '03.

[15]  Markus Weimer,et al.  Learning To Solve Circuit-SAT: An Unsupervised Differentiable Approach , 2018, ICLR.

[16]  Alan M. Frieze,et al.  Random graphs , 2006, SODA '06.

[17]  Demis Hassabis,et al.  A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play , 2018, Science.

[18]  Niklas Sörensson,et al.  An Extensible SAT-solver , 2003, SAT.

[19]  Roberto J. Bayardo,et al.  Using CSP Look-Back Techniques to Solve Real-World SAT Instances , 1997, AAAI/IAAI.

[20]  Jason Weston,et al.  Curriculum learning , 2009, ICML '09.

[21]  Uwe Schöning,et al.  Choosing Probability Distributions for Stochastic Local Search and the Role of Make versus Break , 2012, SAT.

[22]  F. Scarselli,et al.  A new model for learning in graph domains , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[23]  Marc Vinyals,et al.  CNFgen: A Generator of Crafted Benchmarks , 2017, SAT.

[24]  Yoshua Bengio,et al.  Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizon , 2018, Eur. J. Oper. Res..

[25]  David L. Dill,et al.  Learning a SAT Solver from Single-Bit Supervision , 2018, ICLR.

[26]  Krzysztof Czarnecki,et al.  Learning Rate Based Branching Heuristic for SAT Solvers , 2016, SAT.

[27]  Hector J. Levesque,et al.  Generating Hard Satisfiability Problems , 1996, Artif. Intell..

[28]  Alex S. Fukunaga,et al.  Automated Discovery of Local Search Heuristics for Satisfiability Testing , 2008, Evolutionary Computation.

[29]  Zhuwen Li,et al.  Combinatorial Optimization with Graph Convolutional Networks and Guided Tree Search , 2018, NeurIPS.

[30]  Michail G. Lagoudakis,et al.  Learning to Select Branching Rules in the DPLL Procedure for Satisfiability , 2001, Electron. Notes Discret. Math..

[31]  Alex S. Fukunaga,et al.  Evolving Local Search Heuristics for SAT Using Genetic Programming , 2004, GECCO.

[32]  Armin Biere.  Yet another Local Search Solver and Lingeling and Friends Entering the SAT Competition 2014 , 2014 .

[33]  Cesare Tinelli,et al.  Handbook of Satisfiability , 2021, Handbook of Satisfiability.

[34]  Samy Bengio,et al.  Neural Combinatorial Optimization with Reinforcement Learning , 2016, ICLR.

[35]  Krzysztof Czarnecki,et al.  An Empirical Study of Branching Heuristics Through the Lens of Global Learning Rate , 2017, SAT.

[36]  Simon M. Lucas,et al.  A Survey of Monte Carlo Tree Search Methods , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[37]  Carlos Ansótegui,et al.  The Community Structure of SAT Formulas , 2012, SAT.

[38]  Bart Selman,et al.  Local search strategies for satisfiability testing , 1993, Cliques, Coloring, and Satisfiability.

[39]  Sanjit A. Seshia,et al.  Learning Heuristics for Automated Reasoning through Deep Reinforcement Learning , 2018, ArXiv.

[40]  Ah Chung Tsoi,et al.  The Graph Neural Network Model , 2009, IEEE Transactions on Neural Networks.

[41]  Daniel R. Tauritz,et al.  Improving performance of CDCL SAT solvers by automated design of variable selection heuristics , 2017, 2017 IEEE Symposium Series on Computational Intelligence (SSCI).

[42]  Armin Biere,et al.  Stochastic Local Search for Satisfiability Modulo Theories , 2015, AAAI.

[43]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[44]  Ronald J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[45]  Max Welling,et al.  Attention, Learn to Solve Routing Problems! , 2018, ICLR.

[46]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .