DAGN: Discourse-Aware Graph Network for Logical Reasoning

Recent QA with logical reasoning questions requires passage-level relations among the sentences. However, current approaches still focus on sentence-level relations interacting among tokens. In this work, we explore aggregating passage-level clues for solving logical reasoning QA by using discourse-based information. We propose a discourse-aware graph network (DAGN) that reasons relying on the discourse structure of the texts. The model encodes discourse information as a graph with elementary discourse units (EDUs) and discourse relations, and learns the discourse-aware features via a graph network for downstream QA tasks. Experiments are conducted on two logical reasoning QA datasets, ReClor and LogiQA, and our proposed DAGN achieves competitive results. The source code is available at https://github.com/Eleanor-H/DAGN.

[1]  Xiaocheng Feng,et al.  Dialogue Discourse-Aware Graph Convolutional Networks for Abstractive Meeting Summarization , 2020, ArXiv.

[2]  Piji Li,et al.  Consistency and Coherency Enhanced Story Generation , 2020, ECIR.

[3]  Todor Mihaylov,et al.  Discourse-Aware Semantic Self-Attention for Narrative Reading Comprehension , 2019, EMNLP.

[4]  Shafiq R. Joty,et al.  Discern: Discourse-Aware Entailment Reasoning Network for Conversational Machine Reading , 2020, EMNLP.

[5]  Shafiq R. Joty,et al.  Discourse Analysis and Its Applications , 2019, ACL.

[6]  Franck Dernoncourt,et al.  A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents , 2018, NAACL.

[7]  Zhiyuan Liu,et al.  NumNet: Machine Reading Comprehension with Numerical Reasoning , 2019, EMNLP.

[8]  Rashmi Prasad,et al.  The Penn Discourse Treebank , 2004, LREC.

[9]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Masayuki Asahara,et al.  Adversarial Training for Commonsense Inference , 2020, REPL4NLP.

[11]  Rico Sennrich,et al.  Context-Aware Neural Machine Translation Learns Anaphora Resolution , 2018, ACL.

[12]  Geoffrey E. Hinton,et al.  Layer Normalization , 2016, ArXiv.

[13]  Yejin Choi,et al.  Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning , 2019, EMNLP.

[14]  Nicola De Cao,et al.  Question Answering by Reasoning Across Documents with Graph Convolutional Networks , 2018, NAACL.

[15]  Wei Chu,et al.  Question Directed Graph Attention Network for Numerical Reasoning over Text , 2020, EMNLP.

[16]  Peter Clark,et al.  Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering , 2018, EMNLP.

[17]  Joey Tianyi Zhou,et al.  Deep Reinforcement Learning with Stacked Hierarchical Attention for Text-based Games , 2020, NeurIPS.

[18]  Yan Wang,et al.  Graph-to-Tree Learning for Solving Math Word Problems , 2020, ACL.

[19]  Lei Li,et al.  Dynamically Fused Graph Network for Multi-hop Reasoning , 2019, ACL.

[20]  Yu Cheng,et al.  Discourse-Aware Neural Extractive Text Summarization , 2020, ACL.

[21]  Yoshua Bengio,et al.  HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering , 2018, EMNLP.

[22]  Jiashi Feng,et al.  ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning , 2020, ICLR.

[23]  Peter Clark,et al.  WIQA: A dataset for “What if...” reasoning over procedural text , 2019, EMNLP.

[24]  Liang Lin,et al.  REM-Net: Recursive Erasure Memory Network for Commonsense Evidence Refinement , 2021, AAAI.

[25]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[26]  William C. Mann,et al.  Rhetorical Structure Theory: Toward a functional theory of text organization , 1988 .

[27]  Dan Roth,et al.  Looking Beyond the Surface: A Challenge Set for Reading Comprehension over Multiple Sentences , 2018, NAACL.

[28]  Jian Zhang,et al.  SQuAD: 100,000+ Questions for Machine Comprehension of Text , 2016, EMNLP.

[29]  Bhavana Dalvi,et al.  What-if I ask you to explain: Explaining the effects of perturbations in procedural text , 2020, EMNLP.

[30]  Sebastian Riedel,et al.  Constructing Datasets for Multi-hop Reading Comprehension Across Documents , 2017, TACL.

[31]  Livio Robaldo,et al.  The Penn Discourse TreeBank 2.0. , 2008, LREC.

[32]  Hanmeng Liu,et al.  LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning , 2020, IJCAI.

[33]  Yoshua Bengio,et al.  On the Properties of Neural Machine Translation: Encoder–Decoder Approaches , 2014, SSST@EMNLP.

[34]  Gabriel Stanovsky,et al.  DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs , 2019, NAACL.

[35]  Po-Sen Huang,et al.  Discourse-Aware Neural Rewards for Coherent Text Generation , 2018, NAACL.

[36]  Yiming Yang,et al.  EIGEN: Event Influence GENeration using Pre-trained Language Models , 2020, ArXiv.

[37]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[38]  Dan Roth,et al.  “Going on a vacation” takes longer than “Going for a walk”: A Study of Temporal Commonsense Understanding , 2019, EMNLP.

[39]  Kevin Gimpel,et al.  Gaussian Error Linear Units (GELUs) , 2016 .

[40]  Shafiq R. Joty,et al.  Weakly Supervised Neuro-Symbolic Module Networks for Numerical Reasoning , 2021, AAAI.

[41]  Yu Cao,et al.  BAG: Bi-directional Attention Entity Graph Convolutional Network for Multi-hop Reasoning Question Answering , 2019, NAACL.