Scallop: A Language for Neurosymbolic Programming

We present Scallop, a language which combines the benefits of deep learning and logical reasoning. Scallop enables users to write a wide range of neurosymbolic applications and train them in a data- and compute-efficient manner. It achieves these goals through three key features: 1) a flexible symbolic representation that is based on the relational data model; 2) a declarative logic programming language that is based on Datalog and supports recursion, aggregation, and negation; and 3) a framework for automatic and efficient differentiable reasoning that is based on the theory of provenance semirings. We evaluate Scallop on a suite of eight neurosymbolic applications from the literature. Our evaluation demonstrates that Scallop is capable of expressing algorithmic reasoning in diverse and challenging AI tasks, provides a succinct interface for machine learning programmers to integrate logical domain knowledge, and yields solutions that are comparable or superior to state-of-the-art models in terms of accuracy. Furthermore, Scallop's solutions outperform these models in aspects such as runtime and data efficiency, interpretability, and generalizability.

[1]  Noah D. Goodman,et al.  Parsel: A (De-)compositional Framework for Algorithmic Reasoning with Language Models , 2022, 2212.10561.

[2]  Debadeepta Dey,et al.  What Makes Convolutional Models Great on Long Sequence Modeling? , 2022, ICLR.

[3]  Dragomir R. Radev,et al.  Binding Language Models in Symbolic Languages , 2022, ICLR.

[4]  Yuhuai Wu,et al.  Solving Quantitative Reasoning Problems with Language Models , 2022, NeurIPS.

[5]  Christopher Ré,et al.  How to Train Your HiPPO: State Space Models with Generalized Orthogonal Basis Projections , 2022, ICLR.

[6]  Devi Parikh,et al.  MUGEN: A Playground for Video-Audio-Text Multimodal Understanding and GENeration , 2022, ECCV.

[7]  Albert Gu,et al.  Efficiently Modeling Long Sequences with Structured State Spaces , 2021, ICLR.

[8]  Mahmoud Abo Khamis,et al.  Convergence of Datalog over (Pre-) Semirings , 2021, PODS.

[9]  Stephen Muggleton,et al.  Inductive logic programming at 30 , 2021, Machine Learning.

[10]  Mohan S. Kankanhalli,et al.  Don't Pour Cereal into Coffee: Differentiable Temporal Logic for Temporal Action Segmentation , 2022, NeurIPS.

[11]  February Friday.  Evaluating Large Language Models , 2022 .

[12]  Wojciech Zaremba,et al.  Evaluating Large Language Models Trained on Code , 2021, ArXiv.

[13]  Isil Dillig,et al.  Web question answering with neurosymbolic program synthesis , 2021, PLDI.

[14]  Liu Yang,et al.  Long Range Arena: A Benchmark for Efficient Transformers , 2020, ICLR.

[15]  Luc De Raedt,et al.  Neural probabilistic logic programming in DeepProbLog , 2019, Artif. Intell..

[16]  M. Naik,et al.  Scallop: From Probabilistic Deductive Databases to Scalable Differentiable Reasoning , 2021, NeurIPS.

[17]  Val Tannen,et al.  Semiring Provenance for Fixed-Point Logic , 2021, CSL.

[18]  Jennifer J. Sun,et al.  Learning Differentiable Programs with Admissible Neural Heuristics , 2020, NeurIPS.

[19]  Edward Grefenstette,et al.  Learning Reasoning Strategies in End-to-End Differentiable Proving , 2020, ICML.

[20]  Qing Li,et al.  Closed Loop Neural-Symbolic Learning via Integrating Neural Perception, Grammar Parsing, and Symbolic Reasoning , 2020, ICML.

[21]  Nils Gesbert,et al.  On the Optimization of Recursive Relational Queries: Application to Graph Queries , 2020, SIGMOD Conference.

[22]  Mark Chen,et al.  Language Models are Few-Shot Learners , 2020, NeurIPS.

[23]  Quoc V. Le,et al.  Neural Symbolic Reader: Scalable Integration of Distributed and Symbolic Representations for Reading Comprehension , 2020, ICLR.

[24]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[25]  Thomas Wolf,et al.  DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter , 2019, ArXiv.

[26]  Mohit Bansal,et al.  LXMERT: Learning Cross-Modality Encoder Representations from Transformers , 2019, EMNLP.

[27]  Shiguang Shan,et al.  From Two Graphs to N Questions: A VQA Dataset for Compositional Reasoning on Vision and Commonsense , 2019, ArXiv.

[28]  Joelle Pineau,et al.  CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Text , 2019, EMNLP.

[29]  Omer Levy,et al.  RoBERTa: A Robustly Optimized BERT Pretraining Approach , 2019, ArXiv.

[30]  Mukund Raghothaman,et al.  Synthesizing Datalog Programs Using Numerical Relaxation , 2019, IJCAI.

[31]  Priya L. Donti,et al.  SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solver , 2019, ICML.

[32]  Chuang Gan,et al.  The Neuro-Symbolic Concept Learner: Interpreting Scenes Words and Sentences from Natural Supervision , 2019, ICLR.

[33]  Christopher D. Manning,et al.  GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Taehoon Kim,et al.  Quantifying Generalization in Reinforcement Learning , 2018, ICML.

[35]  Noah D. Goodman,et al.  Pyro: Deep Universal Probabilistic Programming , 2018, J. Mach. Learn. Res..

[36]  Chuang Gan,et al.  Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding , 2018, NeurIPS.

[37]  Hongseok Yang,et al.  An Introduction to Probabilistic Programming , 2018, ArXiv.

[38]  Chen Sun,et al.  Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification , 2017, ECCV.

[39]  Guy Van den Broeck,et al.  A Semantic Loss Function for Deep Learning with Symbolic Knowledge , 2017, ICML.

[40]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[41]  Zoubin Ghahramani,et al.  Turing: A Language for Flexible Probabilistic Inference , 2018 .

[42]  Germán Ros,et al.  CARLA: An Open Urban Driving Simulator , 2017, CoRL.

[43]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[44]  Li Fei-Fei,et al.  CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Jian Zhang,et al.  SQuAD: 100,000+ Questions for Machine Comprehension of Text , 2016, EMNLP.

[46]  Till Westmann,et al.  On fast large-scale program analysis in Datalog , 2016, CC.

[47]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[48]  Dan Klein,et al.  Neural Module Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[49]  A. Murawski.  Foundations and Trends R © in Programming Languages , 2016 .

[50]  Luc De Raedt,et al.  ProbLog2: Probabilistic Logic Programming , 2015, ECML/PKDD.

[51]  Michael S. Bernstein,et al.  Image retrieval using scene graphs , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[52]  Emir Pasalic,et al.  Design and Implementation of the LogicBlox System , 2015, SIGMOD Conference.

[53]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[54]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[55]  V. S. Costa,et al.  Inductive Logic Programming , 2014, Lecture Notes in Computer Science.

[56]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[57]  Adnan Darwiche,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence SDD: A New Canonical Representation of Propositional Knowledge Bases , 2022 .

[58]  Daniel Deutch,et al.  On the Limitations of Provenance for Queries with Difference , 2011, TaPP.

[59]  Val Tannen,et al.  Provenance semirings , 2007, PODS.

[60]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[61]  Serge Abiteboul,et al.  Foundations of Databases: The Logical Level , 1995 .

[62]  C. Cordell Green,et al.  What Is Program Synthesis? , 1985, J. Autom. Reason..