Can Graph Neural Networks Learn to Solve MaxSAT Problem?

With the rapid development of deep learning techniques, various recent work has tried to apply graph neural networks (GNNs) to solve NP-hard problems such as Boolean Satisfiability (SAT), which shows the potential in bridging the gap between machine learning and symbolic reasoning. However, the quality of solutions predicted by GNNs has not been well investigated in the literature. In this paper, we study the capability of GNNs in learning to solve Maximum Satisfiability (MaxSAT) problem, both from theoretical and practical perspectives. We build two kinds of GNN models to learn the solution of MaxSAT instances from benchmarks, and show that GNNs have attractive potential to solve MaxSAT problem through experimental evaluation. We also present a theoretical explanation of the effect that GNNs can learn to solve MaxSAT problem to some extent for the first time, based on the algorithmic alignment theory.

[1]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[2]  Hisashi Kashima,et al.  Approximation Ratios of Graph Neural Networks for Combinatorial Problems , 2019, NeurIPS.

[3]  7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 , 2019 .

[4]  Le Song,et al.  Learning to Branch in Mixed Integer Programming , 2016, AAAI.

[5]  Hector J. Levesque,et al.  Hard and Easy Distributions of SAT Problems , 1992, AAAI.

[6]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[7]  Ken-ichi Kawarabayashi,et al.  What Can Neural Networks Reason About? , 2019, ICLR.

[8]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[9]  J. J. Hopfield,et al.  “Neural” computation of decisions in optimization problems , 1985, Biological Cybernetics.

[10]  Shaowei Cai,et al.  Deep Cooperation of CDCL and Local Search for SAT , 2022, SAT.

[11]  Edward A. Lee,et al.  Learning Heuristics for Quantified Boolean Formulas through Reinforcement Learning , 2020, ICLR.

[12]  David S. Johnson,et al.  Some Simplified NP-Complete Graph Problems , 1976, Theor. Comput. Sci..

[13]  Dana Ron,et al.  On Approximating the Minimum Vertex Cover in Sublinear Time and the Connection to Distributed Algorithms , 2007, Electron. Colloquium Comput. Complex..

[14]  Fan Zhang,et al.  Learning the Satisfiability of Pseudo-Boolean Problem with Graph Neural Networks , 2020, CP.

[15]  Luís C. Lamb,et al.  Graph Colouring Meets Deep Learning: Effective Graph Neural Network Models for Combinatorial Problems , 2019, 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI).

[16]  Barnabás Póczos,et al.  Learning Local Search Heuristics for Boolean Satisfiability , 2019, NeurIPS.

[17]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Toby Walsh,et al.  Handbook of Satisfiability: Volume 185 Frontiers in Artificial Intelligence and Applications , 2009 .

[19]  Yoshua Bengio,et al.  Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizon , 2018, Eur. J. Oper. Res..

[20]  Luís C. Lamb,et al.  Learning to Solve NP-Complete Problems - A Graph Neural Network for the Decision TSP , 2018, AAAI.

[21]  S. Biadgilign,et al.  Addis Ababa, Ethiopia , 2019, The Statesman’s Yearbook Companion.

[22]  Jan M. Rabaey,et al.  Distributed algorithms for transmission power control in wireless sensor networks , 2003, 2003 IEEE Wireless Communications and Networking, 2003. WCNC 2003..

[23]  8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020 , 2020, ICLR.

[24]  Andrea Lodi,et al.  Combinatorial optimization and reasoning with graph neural networks , 2021, IJCAI.

[25]  Le Song,et al.  2 Common Formulation for Greedy Algorithms on Graphs , 2018 .

[26]  Maria Luisa Bonet,et al.  SAT-based MaxSAT algorithms , 2013, Artif. Intell..

[27]  Vijay V. Vazirani,et al.  Approximation Algorithms , 2001, Springer Berlin Heidelberg.

[28]  Navdeep Jaitly,et al.  Pointer Networks , 2015, NIPS.

[29]  Moshe Y. Vardi,et al.  Graph Neural Networks Meet Neural-Symbolic Computing: A Survey and Perspective , 2020, IJCAI.

[30]  Kevin Leyton-Brown,et al.  Predicting Propositional Satisfiability via End-to-End Learning , 2020, AAAI.

[31]  David L. Dill,et al.  Learning a SAT Solver from Single-Bit Supervision , 2018, ICLR.

[32]  Zhiyuan Liu,et al.  Graph Neural Networks: A Review of Methods and Applications , 2018, AI Open.

[33]  Mislav Balunovic,et al.  Learning to Solve SMT Formulas , 2018, NeurIPS.

[34]  Michael Elkin,et al.  Distributed approximation: a survey , 2004, SIGA.

[35]  Stephen A. Cook,et al.  The complexity of theorem-proving procedures , 1971, STOC.