Inner Attention Based bi-LSTMs with Indexing for non-Factoid Question Answering

In this paper, we focussed on non-factoid question answering problem using a bidirectional LSTM with an inner attention mechanism and indexing for better accuracy. Non factoid QA is an important task and can be significantly applied in constructing useful knowledge bases and extracting valuable information. The advantage of using Deep Learning frameworks in solving these kind of problems is that it does not require any feature engineering and other linguistic tools. The proposed approach is to extend a LSTM (Long Short Term Memory) model in two directions, one with a Convolutional layer and other with an inner attention mechanism, proposed by Bingning Wang, et al., to the LSTMs, to generate answer representations in accordance with the question. On top of this Deep Learning model we used an information retrieval model based on indexing to generate answers and improve the accuracy. The proposed methodology showed an improvement in accuracy over the referred model and respective baselines and also with respect to the answer lengths used. The models are tested with two non factoid QA data sets: TREC-QA and InsuranceQA.

[1]  Razvan Pascanu,et al.  Theano: new features and speed improvements , 2012, ArXiv.

[2]  Bowen Zhou,et al.  LSTM-based Deep Learning Models for non-factoid answer selection , 2015, ArXiv.

[3]  Jun Zhao,et al.  Inner Attention based Recurrent Neural Networks for Answer Selection , 2016, ACL.

[4]  Christopher D. Manning,et al.  Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.

[5]  Phil Blunsom,et al.  Reasoning about Entailment with Neural Attention , 2015, ICLR.

[6]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[7]  W. Bruce Croft,et al.  End to End Long Short Term Memory Networks for Non-Factoid Question Answering , 2016, ICTIR.

[8]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[9]  Andrew Chou,et al.  Semantic Parsing on Freebase from Question-Answer Pairs , 2013, EMNLP.

[10]  Noah A. Smith,et al.  What is the Jeopardy Model? A Quasi-Synchronous Grammar for QA , 2007, EMNLP.

[11]  Bowen Zhou,et al.  Applying deep learning to answer selection: A study and an open task , 2015, 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU).

[12]  Jason Weston,et al.  End-To-End Memory Networks , 2015, NIPS.

[13]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[14]  Yuxin Peng,et al.  The application of two-level attention models in deep convolutional neural network for fine-grained image classification , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Ramesh Nallapati,et al.  Joint question clustering and relevance prediction for open domain non-factoid question answering , 2014, WWW.

[16]  Phil Blunsom,et al.  Teaching Machines to Read and Comprehend , 2015, NIPS.