Approximation Ratios of Graph Neural Networks for Combinatorial Problems

In this paper, from a theoretical perspective, we study how powerful graph neural networks (GNNs) can be for learning approximation algorithms for combinatorial problems. To this end, we first establish a new class of GNNs that can solve a strictly wider variety of problems than existing GNNs. Then, we bridge the gap between GNN theory and the theory of distributed local algorithms. We theoretically demonstrate that the most powerful GNN can learn approximation algorithms for the minimum dominating set problem and the minimum vertex cover problem with some approximation ratios with the aid of the theory of distributed local algorithms. We also show that most of the existing GNNs such as GIN, GAT, GCN, and GraphSAGE cannot perform better than with these ratios. This paper is the first to elucidate approximation ratios of GNNs for combinatorial problems. Furthermore, we prove that adding coloring or weak-coloring to each node feature improves these approximation ratios. This indicates that preprocessing and feature engineering theoretically strengthen model capabilities.

[1]  Daniel R. Figueiredo,et al.  struc2vec: Learning Node Representations from Structural Identity , 2017, KDD.

[2]  M. Kaufmann What Can Be Computed Locally ? , 2003 .

[3]  Le Song,et al.  2 Common Formulation for Greedy Algorithms on Graphs , 2018 .

[4]  Ah Chung Tsoi,et al.  The Graph Neural Network Model , 2009, IEEE Transactions on Neural Networks.

[5]  Christoph Lenzen,et al.  Local Algorithms: Self-stabilization on Speed , 2009, SSS.

[6]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[7]  Krzysztof Onak,et al.  Constant-Time Approximation Algorithms via Local Improvements , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.

[8]  Zhuwen Li,et al.  Combinatorial Optimization with Graph Convolutional Networks and Guided Tree Search , 2018, NeurIPS.

[9]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[10]  Jan M. Rabaey,et al.  Distributed algorithms for transmission power control in wireless sensor networks , 2003, 2003 IEEE Wireless Communications and Networking, 2003. WCNC 2003..

[11]  Nathan Linial,et al.  Locality in Distributed Graph Algorithms , 1992, SIAM J. Comput..

[12]  Vijay V. Vazirani,et al.  Approximation Algorithms , 2001, Springer Berlin Heidelberg.

[13]  Christoph Lenzen,et al.  Leveraging Linial's Locality Limit , 2008, DISC.

[14]  Jure Leskovec,et al.  Graph Convolutional Neural Networks for Web-Scale Recommender Systems , 2018, KDD.

[15]  Andrzej Czygrinow,et al.  Fast Distributed Approximations in Planar Graphs , 2008, DISC.

[16]  Valentin Polishchuk,et al.  A Local 2-Approximation Algorithm for the Vertex Cover Problem , 2009, DISC.

[17]  F. Scarselli,et al.  A new model for learning in graph domains , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[18]  Lauri Hella,et al.  Weak models of distributed computing, with connections to modal logic , 2012, PODC '12.

[19]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[20]  Ronald J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[21]  Dana Ron,et al.  On Approximating the Minimum Vertex Cover in Sublinear Time and the Connection to Distributed Algorithms , 2007, Electron. Colloquium Comput. Complex..

[22]  Max Welling,et al.  Modeling Relational Data with Graph Convolutional Networks , 2017, ESWC.

[23]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[24]  Samy Bengio,et al.  Neural Combinatorial Optimization with Reinforcement Learning , 2016, ICLR.

[25]  Jukka Suomela,et al.  Survey of local algorithms , 2013, CSUR.

[26]  Valentin Polishchuk,et al.  Local algorithms in (weakly) coloured graphs , 2010, ArXiv.

[27]  Dana Angluin,et al.  Local and global properties in networks of processors (Extended Abstract) , 1980, STOC '80.

[28]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[29]  Roger Wattenhofer,et al.  Distributed Weighted Matching , 2004, DISC.

[30]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[31]  Navdeep Jaitly,et al.  Pointer Networks , 2015, NIPS.