ShadowGNN: Graph Projection Neural Network for Text-to-SQL Parser

Given a database schema, Text-to-SQL aims to translate a natural language question into the corresponding SQL query. Under the setup of cross-domain, traditional semantic parsing models struggle to adapt to unseen database schemas. To improve the model generalization capability for rare and unseen schemas, we propose a new architecture, ShadowGNN, which processes schemas at abstract and semantic levels. By ignoring names of semantic items in databases, abstract schemas are exploited in a well-designed graph projection neural network to obtain delexicalized representation of question and schema. Based on the domain-independent representations, a relation-aware transformer is utilized to further extract logical linking between question and schema. Finally, a SQL decoder with context-free grammar is applied. On the challenging Text-to-SQL benchmark Spider, empirical results show that ShadowGNN outperforms state-of-the-art models. When the annotated data is extremely limited (only 10% training set), ShadowGNN gets over absolute 5% performance gain, which shows its powerful generalization ability. Our implementation will be open-sourced at https://github.com/WowCZ/shadowgnn

[1]  Dong Ryeol Shin,et al.  RYANSQL: Recursively Applying Sketch-based Slot Fillings for Complex Text-to-SQL in Cross-Domain Databases , 2020, CL.

[2]  Zhi Chen,et al.  Policy Adaptation for Deep Reinforcement Learning-Based Dialogue Management , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[3]  Jonathan Berant,et al.  Global Reasoning over Database Structures for Text-to-SQL Parsing , 2019, EMNLP.

[4]  Kai Yu,et al.  LET: Linguistic Knowledge Enhanced Graph Transformer for Chinese Short Text Matching , 2021, AAAI.

[5]  Lu Chen,et al.  Neural Graph Matching Networks for Chinese Short Text Matching , 2020, ACL.

[6]  Max Welling,et al.  Modeling Relational Data with Graph Convolutional Networks , 2017, ESWC.

[7]  Jonathan Berant,et al.  Representing Schema Structure with Graph Neural Networks for Text-to-SQL Parsing , 2019, ACL.

[8]  Mirella Lapata,et al.  Coarse-to-Fine Decoding for Neural Semantic Parsing , 2018, ACL.

[9]  Graham Neubig,et al.  TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data , 2020, ACL.

[10]  Omer Levy,et al.  RoBERTa: A Robustly Optimized BERT Pretraining Approach , 2019, ArXiv.

[11]  Kai Yu,et al.  Semantic Parsing with Dual Learning , 2019, ACL.

[12]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[13]  Graham Neubig,et al.  A Syntactic Neural Model for General-Purpose Code Generation , 2017, ACL.

[14]  Kai Yu,et al.  AgentGraph: Toward Universal Dialogue Management With Structured Deep Reinforcement Learning , 2019, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[15]  Jonathan Berant,et al.  Grammar-based Neural Text-to-SQL Generation , 2019, ArXiv.

[16]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[17]  Yan Gao,et al.  Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representation , 2019, ACL.

[18]  Xiaodong Liu,et al.  RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers , 2020, ACL.

[19]  Luo Si,et al.  Dynamic Hybrid Relation Network for Cross-Domain Context-Dependent Semantic Parsing , 2021, ArXiv.

[20]  Geoffrey E. Hinton,et al.  Layer Normalization , 2016, ArXiv.

[21]  Lu Chen,et al.  Structured Dialogue Policy with Graph Neural Networks , 2018, COLING.

[22]  Tao Yu,et al.  Editing-Based SQL Query Generation for Cross-Domain Context-Dependent Questions , 2019, EMNLP.

[23]  Richard Socher,et al.  Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning , 2018, ArXiv.

[24]  Ashish Vaswani,et al.  Self-Attention with Relative Position Representations , 2018, NAACL.

[25]  Weixin Wang,et al.  Re-examining the Role of Schema Linking in Text-to-SQL , 2020, EMNLP.

[26]  Sida I. Wang,et al.  Grounded Adaptation for Zero-shot Executable Semantic Parsing , 2020, EMNLP.

[27]  Chi Wang,et al.  Schema-Guided Multi-Domain Dialogue State Tracking with Graph Attention Neural Networks , 2020, AAAI.

[28]  Kai Yu,et al.  Distributed Structured Actor-Critic Reinforcement Learning for Universal Dialogue Management , 2020, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[29]  Lu Chen,et al.  Structured Hierarchical Dialogue Policy with Graph Neural Networks , 2020, ArXiv.

[30]  Andrew Chou,et al.  Semantic Parsing on Freebase from Question-Answer Pairs , 2013, EMNLP.

[31]  Yoshua Bengio,et al.  Compositional generalization in a deep seq2seq model by separating syntax and semantics , 2019, ArXiv.

[32]  Lu Chen,et al.  Line Graph Enhanced AMR-to-Text Generation with Mix-Order Graph Attention Networks , 2020, ACL.

[33]  Richard Socher,et al.  Bridging Textual and Tabular Data for Cross-Domain Text-to-SQL Semantic Parsing , 2020, FINDINGS.

[34]  Lu Chen,et al.  Efficient Context and Schema Fusion Networks for Multi-Domain Dialogue State Tracking , 2020, EMNLP.

[35]  Jayant Krishnamurthy,et al.  Neural Semantic Parsing with Type Constraints for Semi-Structured Tables , 2017, EMNLP.

[36]  Yuntao Li,et al.  “What Do You Mean by That?” - a Parser-Independent Interactive Approach for Enhancing Text-to-SQL , 2020, EMNLP.

[37]  Tao Yu,et al.  GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing , 2021, ICLR.

[38]  Marco Baroni,et al.  Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks , 2017, ICML.

[39]  Tao Yu,et al.  Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task , 2018, EMNLP.

[40]  Xiaojun Wan,et al.  IGSQL: Database Schema Interaction Graph Based Neural Model for Context-Dependent Text-to-SQL Generation , 2020, EMNLP.

[41]  Chen Liu,et al.  Unsupervised Dual Paraphrasing for Two-stage Semantic Parsing , 2020, ACL.

[42]  Richard S. Zemel,et al.  Gated Graph Sequence Neural Networks , 2015, ICLR.

[43]  Thomas Wolf,et al.  HuggingFace's Transformers: State-of-the-art Natural Language Processing , 2019, ArXiv.