Graph Neural Networks Meet Neural-Symbolic Computing: A Survey and Perspective

Neural-symbolic computing has now become the subject of interest of both academic and industry research laboratories. Graph Neural Networks (GNN) have been widely used in relational and symbolic domains, with widespread application of GNNs in combinatorial optimization, constraint satisfaction, relational reasoning and other scientific domains. The need for improved explainability, interpretability and trust of AI systems in general demands principled methodologies, as suggested by neural-symbolic computing. In this paper, we review the state-of-the-art on the use of GNNs as a model of neural-symbolic computing. This includes the application of GNNs in several domains as well as its relationship to current developments in neural-symbolic computing.

[1]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[2]  Ryoma Sato,et al.  A Survey on The Expressive Power of Graph Neural Networks , 2020, ArXiv.

[3]  Jason Yosinski,et al.  Hamiltonian Neural Networks , 2019, NeurIPS.

[4]  M. V. Rossum,et al.  In Neural Computation , 2022 .

[5]  Ernest Davis.  The Use of Deep Learning for Symbolic Integration: A Review of (Lample and Charton, 2019) , 2019, ArXiv.

[6]  David L. Dill,et al.  Learning a SAT Solver from Single-Bit Supervision , 2018, ICLR.

[7]  Martin Grohe,et al.  RUN-CSP: Unsupervised Learning of Message Passing Networks for Binary Constraint Satisfaction Problems , 2019, ArXiv.

[8]  Kevin Leyton-Brown,et al.  Predicting Propositional Satisfiability via End-to-End Learning , 2020, AAAI.

[9]  Li Deng,et al.  A Neural-Symbolic Approach to Natural Language Tasks , 2018, ArXiv.

[10]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[11]  Le Song,et al.  2 Common Formulation for Greedy Algorithms on Graphs , 2018 .

[12]  Christopher R'e,et al.  Machine Learning on Graphs: A Model and Comprehensive Taxonomy , 2020, ArXiv.

[13]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Wenwu Zhu,et al.  Deep Learning on Graphs: A Survey , 2018, IEEE Transactions on Knowledge and Data Engineering.

[15]  Marco Gori,et al.  Semantic-based regularization for learning and inference , 2017, Artif. Intell..

[16]  Yoshua Bengio,et al.  Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizon , 2018, Eur. J. Oper. Res..

[17]  Yuan Luo,et al.  Graph Convolutional Networks for Text Classification , 2018, AAAI.

[18]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[19]  Luciano Serafini,et al.  Neural-Symbolic Computing: An Effective Methodology for Principled Integration of Machine Learning and Reasoning , 2019, FLAP.

[20]  Geoffrey E. Hinton,et al.  Using matrices to model symbolic relationship , 2008, NIPS.

[21]  Guo-Wei Wei,et al.  Protein structure prediction beyond AlphaFold , 2019, Nature Machine Intelligence.

[22]  Luc De Raedt,et al.  Neural-Symbolic Learning and Reasoning: Contributions and Challenges , 2015, AAAI Spring Symposia.

[23]  Anima Anandkumar,et al.  Combining Symbolic Expressions and Black-box Function Evaluations in Neural Programs , 2018, ICLR.

[24]  Tim Rocktäschel,et al.  Learning Knowledge Base Inference with Neural Theorem Provers , 2016, AKBC@NAACL-HLT.

[25]  Navdeep Jaitly,et al.  Pointer Networks , 2015, NIPS.

[26]  Artur S. d'Avila Garcez,et al.  Reasoning about Time and Knowledge in Neural Symbolic Learning Systems , 2003, NIPS.

[27]  Pascal Hitzler,et al.  Logic programs and connectionist networks , 2004, J. Appl. Log..

[28]  Max Welling,et al.  Attention, Learn to Solve Routing Problems! , 2018, ICLR.

[29]  Luís C. Lamb,et al.  Graph Colouring Meets Deep Learning: Effective Graph Neural Network Models for Combinatorial Problems , 2019, 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI).

[30]  Richard S. Zemel,et al.  Gated Graph Sequence Neural Networks , 2015, ICLR.

[31]  Jürgen Schmidhuber,et al.  Relational Neural Expectation Maximization: Unsupervised Discovery of Objects and their Interactions , 2018, ICLR.

[32]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[33]  Luís C. Lamb,et al.  Learning to Solve NP-Complete Problems - A Graph Neural Network for the Decision TSP , 2018, AAAI.

[34]  Graham Neubig,et al.  Learning to Represent Edits , 2018, ICLR.

[35]  Jure Leskovec,et al.  G2SAT: Learning to Generate SAT Formulas , 2019, NeurIPS.

[36]  Richard Evans,et al.  Learning Explanatory Rules from Noisy Data , 2017, J. Artif. Intell. Res..

[37]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[38]  Andrew G. Glen,et al.  APPL , 2001 .

[39]  Joan Bruna,et al.  Few-Shot Learning with Graph Neural Networks , 2017, ICLR.

[40]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[41]  Joe Townsend,et al.  Extracting Relational Explanations From Deep Neural Networks: A Survey From a Neural-Symbolic Perspective , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[42]  Emma J. Chory,et al.  A Deep Learning Approach to Antibiotic Discovery , 2020, Cell.

[43]  Luc De Raedt,et al.  DeepProbLog: Neural Probabilistic Logic Programming , 2018, BNAIC/BENELEARN.

[44]  Gary Marcus,et al.  The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence , 2020, ArXiv.

[45]  Jason Weston,et al.  Learning Structured Embeddings of Knowledge Bases , 2011, AAAI.

[46]  Thomas Lukasiewicz,et al.  Learning to Reason: Leveraging Neural Networks for Approximate DNF Counting , 2019, AAAI.

[47]  Razvan Pascanu,et al.  A simple neural network module for relational reasoning , 2017, NIPS.

[48]  Chuang Gan,et al.  The Neuro-Symbolic Concept Learner: Interpreting Scenes Words and Sentences from Natural Supervision , 2019, ICLR.

[49]  Sergey Levine,et al.  Recurrent Independent Mechanisms , 2019, ICLR.

[50]  Razvan Pascanu,et al.  Relational inductive biases, deep learning, and graph networks , 2018, ArXiv.

[51]  R. Rosenfeld.  Nature , 2009, Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery.

[52]  Zhuwen Li,et al.  Combinatorial Optimization with Graph Convolutional Networks and Guided Tree Search , 2018, NeurIPS.

[53]  Oleksandr Polozov,et al.  Generative Code Modeling with Graphs , 2018, ICLR.

[54]  Paolo Torroni,et al.  Neural-Symbolic Argumentation Mining: An Argument in Favor of Deep Learning and Reasoning , 2019, Frontiers in Big Data.

[55]  Ah Chung Tsoi,et al.  The Graph Neural Network Model , 2009, IEEE Transactions on Neural Networks.

[56]  Artur S. d'Avila Garcez,et al.  Logic Tensor Networks: Deep Learning and Logical Reasoning from Data and Knowledge , 2016, NeSy@HLAI.

[57]  Dov M. Gabbay,et al.  Neural-Symbolic Cognitive Reasoning , 2008, Cognitive Technologies.

[58]  Guillaume Lample,et al.  Deep Learning for Symbolic Mathematics , 2019, ICLR.