When Retriever-Reader Meets Scenario-Based Multiple-Choice Questions

Scenario-based question answering (SQA) requires retrieving and reading paragraphs from a large corpus to answer a question which is contextualized by a long scenario description. Since a scenario contains both keyphrases for retrieval and much noise, retrieval for SQA is extremely difficult. Moreover, it can hardly be supervised due to the lack of relevance labels of paragraphs for SQA. To meet the challenge, in this paper we propose a joint retriever-reader model called JEEVES where the retriever is implicitly supervised only using QA labels via a novel word weighting mechanism. JEEVES significantly outperforms a variety of strong baselines on multiple-choice questions in three SQA datasets.

[1]  Jason Weston,et al.  Reading Wikipedia to Answer Open-Domain Questions , 2017, ACL.

[2]  Evgeny Kharlamov,et al.  Enriching Documents with Compact, Representative, Relevant Knowledge Graphs , 2020, ArXiv.

[3]  Ming-Wei Chang,et al.  Latent Retrieval for Weakly Supervised Open Domain Question Answering , 2019, ACL.

[4]  Alessandro Moschitti,et al.  Reranking for Efficient Transformer-based Answer Selection , 2020, SIGIR.

[5]  Yuzhong Qu,et al.  CKGG: A Chinese Knowledge Graph for High-School Geography Education and Beyond , 2021, SEMWEB.

[6]  Weijia Jia,et al.  Legal Judgment Prediction via Multi-Perspective Bi-Feedback Network , 2019, IJCAI.

[7]  Dilek Z. Hakkani-Tür,et al.  MMM: Multi-stage Multi-task Learning for Multi-choice Reading Comprehension , 2020, AAAI.

[8]  Zhiyuan Liu,et al.  Legal Judgment Prediction via Topological Learning , 2018, EMNLP.

[9]  Ming-Wei Chang,et al.  Natural Questions: A Benchmark for Question Answering Research , 2019, TACL.

[10]  Wei Zhang,et al.  R3: Reinforced Ranker-Reader for Open-Domain Question Answering , 2018, AAAI.

[11]  Gong Cheng,et al.  GeoSQA: A Benchmark for Scenario-based Question Answering in the Geography Domain at High School Level , 2019, EMNLP.

[12]  Ion Androutsopoulos,et al.  Neural Legal Judgment Prediction in English , 2019, ACL.

[13]  Wanxiang Che,et al.  Pre-Training with Whole Word Masking for Chinese BERT , 2019, ArXiv.

[14]  Danqi Chen,et al.  Dense Passage Retrieval for Open-Domain Question Answering , 2020, EMNLP.

[15]  Jian Zhang,et al.  SQuAD: 100,000+ Questions for Machine Comprehension of Text , 2016, EMNLP.

[16]  Dong Yu,et al.  Evidence Sentence Extraction for Machine Reading Comprehension , 2019, CoNLL.

[17]  Jiancheng Lv,et al.  RikiNet: Reading Wikipedia Pages for Natural Question Answering , 2020, ACL.

[18]  Siddharth Patwardhan,et al.  WatsonPaths: Scenario-Based Question Answering and Inference over Unstructured Information , 2017, AI Mag..

[19]  Yu Sun,et al.  ERNIE: Enhanced Representation through Knowledge Integration , 2019, ArXiv.

[20]  Oren Etzioni,et al.  Combining Retrieval, Statistics, and Inference to Answer Elementary Science Questions , 2016, AAAI.

[21]  Traian Rebedea,et al.  Answering questions by learning to rank - Learning to rank by answering questions , 2019, EMNLP/IJCNLP.

[22]  Yuzhong Qu,et al.  Taking Up the Gaokao Challenge: An Information Retrieval Approach , 2016, IJCAI.

[23]  Zhuosheng Zhang,et al.  SG-Net: Syntax-Guided Machine Reading Comprehension , 2019, AAAI.

[24]  Xiao Li,et al.  TSQA: Tabular Scenario Based Question Answering , 2021, AAAI.

[25]  Jun Zhao,et al.  Dual Head-wise Coattention Network for Machine Comprehension with Multiple-Choice Questions , 2020, CIKM.

[26]  Richard Socher,et al.  Coarse-grain Fine-grain Coattention Network for Multi-evidence Question Answering , 2019, ICLR.

[27]  M. Zaharia,et al.  ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT , 2020, SIGIR.

[28]  Yuzhong Qu,et al.  Towards Answering Geography Questions in Gaokao: A Hybrid Approach , 2018, CCKS.

[29]  Guokun Lai,et al.  RACE: Large-scale ReAding Comprehension Dataset From Examinations , 2017, EMNLP.

[30]  Yuzhong Qu,et al.  Reading Comprehension with Graph-based Temporal-Casual Reasoning , 2018, COLING.

[31]  Hai Zhao,et al.  Dual Co-Matching Network for Multi-choice Reading Comprehension , 2020, AAAI.

[32]  Xiaoyan Wang,et al.  Distinguish Confusing Law Articles for Legal Judgment Prediction , 2020, ACL.