Incorporating Domain Knowledge and Semantic Information into Language Models for Commonsense Question Answering

Commonsense question answering (CSQA) aims to answer questions which require the system to understand related commonsense knowledge that is not explicitly expressed in the given context. Recent advance in neural language models (e.g., BERT) that are pre-trained on a large-scale text corpus and fine-tuned on downstream tasks has boosted the performance on CSQA. However, due to the lack of domain knowledge (e.g., in social situations), these models fail to reason about specific tasks. In this work, we propose an approach to incorporate domain knowledge and semantic information into language model based approaches for better understanding the related commonsense knowledge. Firstly, we extract the knowledge from existing resources by jointly learning to ask and answer as well as semantic role labeling based answering. These two tasks are correlated and can reinforce each other to discover the domain knowledge. Then, we utilize Semantic Role Labeling to enable the system to gain a better understanding of relations among relevant entities. Experimental results on several CSQA benchmarks demonstrate the effectiveness of the proposed approach.