Attentive Recurrent Tensor Model for Community Question Answering

A major challenge to the problem of community question answering is the lexical and semantic gap between the sentence representations. Some solutions to minimize this gap includes the introduction of extra parameters to deep models or augmenting the external handcrafted features. In this paper, we propose a novel attentive recurrent tensor network for solving the lexical and semantic gap in community question answering. We introduce token-level and phrase-level attention strategy that maps input sequences to the output using trainable parameters. Further, we use the tensor parameters to introduce a 3-way interaction between question, answer and external features in vector space. We introduce simplified tensor matrices with L2 regularization that results in smooth optimization during training. The proposed model achieves state-of-the-art performance on the task of answer sentence selection (TrecQA and WikiQA datasets) while outperforming the current state-of-the-art on the tasks of best answer selection (Yahoo! L4) and answer triggering task (WikiQA).

[1]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[2]  Alessandro Moschitti,et al.  Learning to Rank Short Text Pairs with Convolutional Deep Neural Networks , 2015, SIGIR.

[3]  Proceedings of the 25th ACM International on Conference on Information and Knowledge Management , 2016 .

[4]  Yonatan Belinkov,et al.  SLS at SemEval-2016 Task 3: Neural-based Approaches for Ranking in Community Question Answering , 2016, *SEMEVAL.

[5]  Lei Yu,et al.  Deep Learning for Answer Sentence Selection , 2014, ArXiv.

[6]  Bowen Zhou,et al.  Improved Representation Learning for Question Answer Matching , 2016, ACL.

[7]  Bowen Zhou,et al.  ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs , 2015, TACL.

[8]  Preslav Nakov,et al.  SemanticZ at SemEval-2016 Task 3: Ranking Relevant Answers in Community Question Answering Using Semantic Similarity Based on Fine-tuned Word Embeddings , 2016, *SEMEVAL.

[9]  Quan Hung Tran,et al.  JAIST: Combining multiple features for Answer Selection in Community Question Answering , 2015, *SEMEVAL.

[10]  Jimmy J. Lin,et al.  Noise-Contrastive Estimation for Answer Selection with Deep Neural Networks , 2016, CIKM.

[11]  Phil Blunsom,et al.  Teaching Machines to Read and Comprehend , 2015, NIPS.

[12]  Phil Blunsom,et al.  Neural Variational Inference for Text Processing , 2015, ICML.

[13]  Siu Cheung Hui,et al.  Learning to Rank Question Answer Pairs with Holographic Dual LSTM Architecture , 2017, SIGIR.

[14]  Zhiguo Wang,et al.  Sentence Similarity Learning by Lexical Decomposition and Composition , 2016, COLING.

[15]  Ben Carterette,et al.  Proceedings of the 2016 ACM International Conference on the Theory of Information Retrieval , 2016, ICTIR 2016.

[16]  Jimmy J. Lin,et al.  Pairwise Word Interaction Modeling with Deep Neural Networks for Semantic Similarity Measurement , 2016, NAACL.

[17]  W. Bruce Croft,et al.  On the Benefit of Incorporating External Features in a Neural Architecture for Answer Sentence Selection , 2017, SIGIR.

[18]  Xuanjing Huang,et al.  Convolutional Neural Tensor Network Architecture for Community-Based Question Answering , 2015, IJCAI.

[19]  Yi Yang,et al.  WikiQA: A Challenge Dataset for Open-Domain Question Answering , 2015, EMNLP.

[20]  Danqi Chen,et al.  Reasoning With Neural Tensor Networks for Knowledge Base Completion , 2013, NIPS.

[21]  Jimmy J. Lin,et al.  Multi-Perspective Sentence Similarity Modeling with Convolutional Neural Networks , 2015, EMNLP.

[22]  W. Bruce Croft,et al.  Beyond Factoid QA: Effective Methods for Non-factoid Answer Sentence Retrieval , 2016, ECIR.

[23]  Yonatan Belinkov,et al.  VectorSLU: A Continuous Word Vector Approach to Answer Selection in Community Question Answering Systems , 2015, *SEMEVAL.

[24]  W. Bruce Croft,et al.  End to End Long Short Term Memory Networks for Non-Factoid Question Answering , 2016, ICTIR.