Zero-Shot Slot Filling via Latent Question Representation and Reading Comprehension

Slot filling is a demanding task of knowledge base population, which aims to extract facts about particular entities from unstructured text automatically. Most of the existing approaches rely on pre-trained extraction models which may suffer from robustness caused by unseen slots, or the so-called zero-shot slot filling problem. Recent studies try to reduce the slot filling to a machine reading comprehension task and achieve certain improvements on unseen slots, but they still face challenges to generate appropriate questions for models and find the right answers. In this paper, we propose a novel end-to-end approach to address the zero-shot slot filling by unifying the natural language question generation and machine reading comprehension. Especially, we explore how to learn a well-organized latent question representation by incorporating external knowledge. We conduct extensive experiments to validate the effectiveness of our model. Experimental results show that the proposed approach outperforms the state-of-the-art baseline methods in zero-shot scenarios.

[1]  Jianfeng Gao,et al.  A Human Generated MAchine Reading COmprehension Dataset , 2018 .

[2]  Ming Zhou,et al.  Question Generation for Question Answering , 2017, EMNLP.

[3]  Junji Tomita,et al.  Retrieve-and-Read: Multi-task Learning of Information Retrieval and Reading Comprehension , 2018, CIKM.

[4]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[5]  Frederick Reiss,et al.  Rule-Based Information Extraction is Dead! Long Live Rule-Based Information Extraction Systems! , 2013, EMNLP.

[6]  Heike Adel,et al.  Comparing Convolutional Neural Networks to Traditional Models for Slot Filling , 2016, NAACL.

[7]  Ali Farhadi,et al.  Bidirectional Attention Flow for Machine Comprehension , 2016, ICLR.

[8]  Heng Ji,et al.  Improving Slot Filling Performance with Attentive Neural Networks on Dependency Structures , 2017, EMNLP.

[9]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[10]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[11]  Heng Ji,et al.  Unsupervised Person Slot Filling based on Graph Mining , 2016, ACL.

[12]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[13]  Omer Levy,et al.  Zero-Shot Relation Extraction via Reading Comprehension , 2017, CoNLL.

[14]  Honglak Lee,et al.  Learning Structured Output Representation using Deep Conditional Generative Models , 2015, NIPS.

[15]  Vittorio Castelli,et al.  Slot Filling through Statistical Processing and Inference Rules , 2009, TAC.

[16]  Ralph Grishman,et al.  New York University 2012 System for KBP Slot Filling , 2012, TAC.

[17]  Jian Zhang,et al.  SQuAD: 100,000+ Questions for Machine Comprehension of Text , 2016, EMNLP.