A Neural Network Approach to Quote Recommendation in Writings

Quote is a language phenomenon of transcribing the saying of someone else. Proper usage of quote can usually make the statement more elegant and convincing. However, the ability of quote usage is usually limited by the amount of quotes one remembers or knows. Quote recommendation is a task of exploiting abundant quote repositories to help people make better use of quotes while writing. The task is different from conventional recommendation tasks due to the characteristic of quote. A pilot study has explored this task by using a learning to rank framework and manually designed features. However, it is still hard to model the meaning of a quote, which is an interesting and challenging problem. In this paper, we propose a neural network approach based on LSTMs to the quote recommendation task. We directly learn the distributed meaning representations for the contexts and the quotes, and then measure the relevance based on the meaning representations. In particular, we try to represent the words in quotes with specific embeddings, according to the contexts, topics and even author preferences of the quotes. Experimental results on a large dataset show that our proposed approach achieves the state-of-the-art performance and it outperforms several strong baselines.

[1]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[2]  Wenpeng Yin,et al.  Convolutional Neural Network for Paraphrase Identification , 2015, NAACL.

[3]  Thorsten Joachims,et al.  Identifying the original contribution of a document via language modeling , 2009, ECML/PKDD.

[4]  Alessandro Moschitti,et al.  Learning to Rank Short Text Pairs with Convolutional Deep Neural Networks , 2015, SIGIR.

[5]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[6]  Hongfei Yan,et al.  Recommending citations with translation model , 2011, CIKM '11.

[7]  Xiaojun Wan,et al.  Learning to Recommend Quotes for Writing , 2015, AAAI.

[8]  Eduard H. Hovy,et al.  A Model of Coherence Based on Distributed Sentence Representation , 2014, EMNLP.

[9]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[10]  Jeffrey Pennington,et al.  Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection , 2011, NIPS.

[11]  Ellen M. Voorhees,et al.  The TREC-8 Question Answering Track Report , 1999, TREC.

[12]  Rabab Kreidieh Ward,et al.  Deep Sentence Embedding Using Long Short-Term Memory Networks: Analysis and Application to Information Retrieval , 2015, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[13]  Baobao Chang,et al.  Max-Margin Tensor Neural Network for Chinese Word Segmentation , 2014, ACL.

[14]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[15]  Jason Weston,et al.  A Neural Attention Model for Abstractive Sentence Summarization , 2015, EMNLP.

[16]  Daniel Kifer,et al.  Context-aware citation recommendation , 2010, WWW '10.

[17]  Wenpeng Yin,et al.  MultiGranCNN: An Architecture for General Matching of Text Chunks on Multiple Levels of Granularity , 2015, ACL.

[18]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.

[19]  Quoc V. Le,et al.  Distributed Representations of Sentences and Documents , 2014, ICML.

[20]  Xiaojun Wan,et al.  Cross-language context-aware citation recommendation in scientific articles , 2014, SIGIR.

[21]  Razvan Pascanu,et al.  Theano: A CPU and GPU Math Compiler in Python , 2010, SciPy.

[22]  Alex Graves,et al.  Generating Sequences With Recurrent Neural Networks , 2013, ArXiv.

[23]  Rabab K. Ward,et al.  Deep Sentence Embedding Using Long Short-Term Memory Networks; arXiv:1502.06922 , 2015 .

[24]  Xueqi Cheng,et al.  A Deep Architecture for Semantic Matching with Multiple Positional Sentence Representations , 2015, AAAI.

[25]  Daniel Jurafsky,et al.  A Hierarchical Neural Autoencoder for Paragraphs and Documents , 2015, ACL.

[26]  Sanja Fidler,et al.  Skip-Thought Vectors , 2015, NIPS.

[27]  Phil Blunsom,et al.  A Convolutional Neural Network for Modelling Sentences , 2014, ACL.

[28]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[29]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[30]  Wenyi Huang,et al.  Recommending citations: translating papers into references , 2012, CIKM.

[31]  Wenyi Huang,et al.  A Neural Probabilistic Model for Context Based Citation Recommendation , 2015, AAAI.

[32]  Quoc V. Le,et al.  Addressing the Rare Word Problem in Neural Machine Translation , 2014, ACL.

[33]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[34]  Hang Li,et al.  Convolutional Neural Network Architectures for Matching Natural Language Sentences , 2014, NIPS.