Injecting Numerical Reasoning Skills into Knowledge Base Question Answering Models

Embedding-based methods are popular for Knowledge Base Question Answering (KBQA), but few current models have numerical reasoning skills and thus struggle to answer ordinal constrained questions. This paper proposes a new embedding-based KBQA framework which particularly takes numerical reasoning into account. We present NumericalTransformer on top of NSM, a state-of-the-art embedding-based KBQA model, to create NT-NSM. To enable better training, we propose two pre-training tasks with explicit numerical-oriented loss functions on two generated training datasets and a template-based data augmentation method for enriching ordinal constrained QA dataset. Extensive experiments on KBQA benchmarks demonstrate that with the help of our training algorithm, NT-NSM is empowered with numerical reasoning skills and substantially outperforms the baselines in answering ordinal constrained questions.

[1]  Ruslan Salakhutdinov,et al.  Open Domain Question Answering Using Early Fusion of Knowledge Bases and Text , 2018, EMNLP.

[2]  Ji-Rong Wen,et al.  Improving Multi-hop Knowledge Base Question Answering by Learning Intermediate Supervision Signals , 2021, WSDM.

[3]  Sameer Singh,et al.  Do NLP Models Know Numbers? Probing Numeracy in Embeddings , 2019, EMNLP.

[4]  D. Roth,et al.  Do Language Embeddings capture Scales? , 2020, FINDINGS.

[5]  Jonathan Berant,et al.  Semantic Parsing via Paraphrasing , 2014, ACL.

[6]  Hanwang Zhang,et al.  KQA Pro: A Large Diagnostic Dataset for Complex Question Answering over Knowledge Base , 2020, ArXiv.

[7]  Jonathan Berant,et al.  The Web as a Knowledge-Base for Answering Complex Questions , 2018, NAACL.

[8]  Alexander G. Gray,et al.  Leveraging Abstract Meaning Representation for Knowledge Base Question Answering , 2020, FINDINGS.

[9]  Ming-Wei Chang,et al.  The Value of Semantic Parse Labeling for Knowledge Base Question Answering , 2016, ACL.

[10]  Apoorv Saxena,et al.  Improving Multi-hop Question Answering over Knowledge Graphs using Knowledge Base Embeddings , 2020, ACL.

[11]  Taher H. Haveliwala.  Topic-sensitive PageRank , 2002, IEEE Trans. Knowl. Data Eng..

[12]  Omer Levy,et al.  RoBERTa: A Robustly Optimized BERT Pretraining Approach , 2019, ArXiv.

[13]  Quoc V. Le,et al.  QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension , 2018, ICLR.

[14]  Ji-Rong Wen,et al.  A Survey on Complex Knowledge Base Question Answering: Methods, Challenges and Solutions , 2021, IJCAI.

[15]  Tiejun Zhao,et al.  Constraint-Based Question Answering with Knowledge Graph , 2016, COLING.

[16]  Jens Lehmann,et al.  LC-QuAD 2.0: A Large Dataset for Complex Question Answering over Wikidata and DBpedia , 2019, SEMWEB.

[17]  Chen Liang,et al.  Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision , 2016, ACL.

[18]  Yunshi Lan,et al.  Query Graph Generation for Answering Multi-hop Complex Questions from Knowledge Bases , 2020, ACL.

[19]  Mohammed J. Zaki,et al.  Bidirectional Attentive Memory Networks for Question Answering over Knowledge Bases , 2019, NAACL.

[20]  Xirui Ke,et al.  Neural, symbolic and neural-symbolic reasoning on knowledge graphs , 2021, AI Open.

[21]  Ming-Wei Chang,et al.  Semantic Parsing via Staged Query Graph Generation: Question Answering with Knowledge Base , 2015, ACL.

[22]  Andrew Chou,et al.  Semantic Parsing on Freebase from Question-Answer Pairs , 2013, EMNLP.

[23]  Jason Weston,et al.  Key-Value Memory Networks for Directly Reading Documents , 2016, EMNLP.

[24]  Lingling Zhang,et al.  SPARQA: Skeleton-based Semantic Parsing for Complex Questions over Knowledge Bases , 2020, AAAI.

[25]  Jonathan Berant,et al.  Injecting Numerical Reasoning Skills into Language Models , 2020, ACL.

[26]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.