M-SQL: Multi-Task Representation Learning for Single-Table Text2sql Generation

Text2SQL can help non-professionals connect with databases by turning natural languages into SQL. Although previous researches about Text2SQL have provided some workable solutions, most of them extract values based on column representation. If there are multiple values in the query and these values belong to different columns, the previous approaches based on column representation cannot accurately extract values. In this work, we propose a new neural network architecture based on the pre-trained BERT, called M-SQL. The column-based value extraction is divided into two modules, value extraction and value-column matching. We evaluate M-SQL on a more complicated TableQA dataset, which comes from an AI competition. We rank first in this competition. Experimental results and competition ranking show that our proposed M-SQL achieves state-of-the-art results on TableQA.

[1]  Xiaodong Liu,et al.  Multi-Task Deep Neural Networks for Natural Language Understanding , 2019, ACL.

[2]  Peter Thanisch,et al.  MASQUE/SQL: an efficient and portable natural language query interface for relational databases , 1993 .

[3]  Mirella Lapata,et al.  Language to Logical Form with Neural Attention , 2016, ACL.

[4]  Richard Socher,et al.  The Natural Language Decathlon: Multitask Learning as Question Answering , 2018, ArXiv.

[5]  Sonal Gupta,et al.  Semantic Parsing for Task Oriented Dialog using Hierarchical Representations , 2018, EMNLP.

[6]  Kaushik Chakrabarti,et al.  X-SQL: reinforce schema representation with context , 2019, ArXiv.

[7]  Richard Socher,et al.  Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning , 2018, ArXiv.

[8]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[9]  Wanxiang Che,et al.  Pre-Training with Whole Word Masking for Chinese BERT , 2019, ArXiv.

[10]  Dawn Xiaodong Song,et al.  SQLNet: Generating Structured Queries From Natural Language Without Reinforcement Learning , 2017, ArXiv.

[11]  Seunghyun Park,et al.  A Comprehensive Exploration on WikiSQL with Table-Aware Word Contextualization , 2019, ArXiv.

[12]  Sergei Gorlatch,et al.  High performance stencil code generation with Lift , 2018, CGO.

[13]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[14]  Mirella Lapata,et al.  Coarse-to-Fine Decoding for Neural Semantic Parsing , 2018, ACL.

[15]  Ming Zhou,et al.  Dialog-to-Action: Conversational Question Answering Over a Large-Scale Knowledge Base , 2018, NeurIPS.

[16]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[17]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[18]  Weizhu Chen,et al.  IncSQL: Training Incremental Text-to-SQL Parsers with Non-Deterministic Oracles , 2018, ArXiv.

[19]  Alec Radford,et al.  Improving Language Understanding by Generative Pre-Training , 2018 .

[20]  Shuohang Wang,et al.  Machine Comprehension Using Match-LSTM and Answer Pointer , 2016, ICLR.

[21]  Tao Yu,et al.  TypeSQL: Knowledge-Based Type-Aware Neural Text-to-SQL Generation , 2018, NAACL.

[22]  Percy Liang,et al.  Data Recombination for Neural Semantic Parsing , 2016, ACL.

[23]  Rishabh Singh,et al.  Robust Text-to-SQL Generation with Execution-Guided Decoding , 2018, 1807.03100.

[24]  David H. D. Warren,et al.  An Efficient Easily Adaptable System for Interpreting Natural Language Queries , 1982, CL.

[25]  Sreela Sasi,et al.  Ontology-based Sentiment Analysis Process for Social Media Content , 2015, INNS Conference on Big Data.

[26]  Luke S. Zettlemoyer,et al.  Deep Contextualized Word Representations , 2018, NAACL.

[27]  Yue Zhang,et al.  Design Challenges and Misconceptions in Neural Sequence Labeling , 2018, COLING.