Identifying facts for chatbot's question answering via sequence labelling using recurrent neural networks

Question answering (QA) is one of the most significant tasks in artificial intelligence (AI) that not yet solved completely. It is a task which requires building data models capable of providing answers to questions expressed in human language. Full question answering involves some form of labelling. Sequence labelling is an important problem in natural language processing. Given a sequence of words, sequence tagging aims to predict a linguistic tag for each word such as the POS tag. The main question is, how can we leverage the sequential nature of data to extract better features and structural dependency between them. In an effort to address this question, this study present attention-based architecture for sequence labelling on deep recurrent neural network (DRNN). To solve the QA problem, it consists of two major steps: identifying the fact and answering the question based on the relevant fact. The study showed that the proposed model provides consistent improvement and outperform then traditional approaches.

[1]  Phil Blunsom,et al.  Recurrent Continuous Translation Models , 2013, EMNLP.

[2]  Isabelle Tellier,et al.  New Recurrent Neural Network Variants for Sequence Labeling , 2016, CICLing.

[3]  Doug Downey,et al.  VecShare: A Framework for Sharing Word Representation Vectors , 2017, EMNLP.

[4]  Eduard H. Hovy,et al.  End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF , 2016, ACL.

[5]  Dan Roth,et al.  Design Challenges and Misconceptions in Named Entity Recognition , 2009, CoNLL.

[6]  Alex Graves,et al.  Supervised Sequence Labelling with Recurrent Neural Networks , 2012, Studies in Computational Intelligence.

[7]  Le-Minh Nguyen,et al.  Semantic Refinement GRU-Based Neural Language Generation for Spoken Dialogue Systems , 2017, PACLING.

[8]  Zaiqing Nie,et al.  Joint Entity Recognition and Disambiguation , 2015, EMNLP.

[9]  Guillaume Lample,et al.  Neural Architectures for Named Entity Recognition , 2016, NAACL.

[10]  Yuji Matsumoto,et al.  Chunking with Support Vector Machines , 2001, NAACL.

[11]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[12]  Shachar Mirkin,et al.  Joint Learning of Correlated Sequence Labeling Tasks Using Bidirectional Recurrent Neural Networks , 2017, INTERSPEECH.

[13]  Huizhen Wang,et al.  A Retrieval-Based Matching Approach to Open Domain Knowledge-Based Question Answering , 2017, NLPCC.

[14]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[15]  Noah A. Smith,et al.  Transition-Based Dependency Parsing with Stack Long Short-Term Memory , 2015, ACL.

[16]  Lukás Burget,et al.  Recurrent neural network based language model , 2010, INTERSPEECH.

[17]  Richard Socher,et al.  A Neural Network for Factoid Question Answering over Paragraphs , 2014, EMNLP.

[18]  Ruofei Zhang,et al.  DeepProbe: Information Directed Sequence Understanding and Chatbot Design via Recurrent Neural Networks , 2017, KDD.

[19]  Eric Nichols,et al.  Named Entity Recognition with Bidirectional LSTM-CNNs , 2015, TACL.

[20]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[21]  Xiang Ren,et al.  Empower Sequence Labeling with Task-Aware Neural Language Model , 2017, AAAI.

[22]  Jason Weston,et al.  Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks , 2015, ICLR.

[23]  Siu-Ming Yiu,et al.  Relevant Fact Selection for QA via Sequence Labeling , 2017, KSEM.

[24]  Giuseppe Riccardi,et al.  Generative and discriminative algorithms for spoken language understanding , 2007, INTERSPEECH.

[25]  Robert E. Mercer,et al.  Improving Neural Sequence Labelling Using Additional Linguistic Information , 2018, 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA).

[26]  Isabelle Tellier,et al.  Improving Recurrent Neural Networks For Sequence Labelling , 2016, ArXiv.

[27]  Iryna Gurevych,et al.  Reporting Score Distributions Makes a Difference: Performance Study of LSTM-networks for Sequence Tagging , 2017, EMNLP.

[28]  Gang Luo,et al.  Joint Named Entity Recognition and Disambiguation , 2015 .