CACV-tree: A New Computational Approach for Sentence Similarity Modeling

Sentence similarity modeling plays an important role in Natural Language Processing (NLP) tasks, and thus has received much attention. In recent years, due to the success of word embedding, the neural network method has achieved sentence embedding, obtaining attractive performance. Nevertheless, most of them focused on learning semantic information and modeling it as a continuous vector, while the syntactic information of sentences has not been fully exploited. On the other hand, prior works have shown the benefits of structured trees that include syntactic information, while few methods in this branch utilized the advantages of sentence compression. This paper makes the first attempt to absorb their advantages by merging these techniques in a unified structure, dubbed as CACV-tree (Compression Attention Constituency Vector-tree). The experimental results, based on 14 widely used datasets, demonstrate that our model is effective and competitive, compared against state-of-the-art models.

[1]  Walter Daelemans,et al.  Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) , 2014, EMNLP 2014.

[2]  Di Wang,et al.  A Long Short-Term Memory Model for Answer Sentence Selection in Question Answering , 2015, ACL.

[3]  Joachim Bingel,et al.  Weakly Supervised Part-of-speech Tagging Using Eye-tracking Data , 2016, ACL.

[4]  Alessandro Moschitti,et al.  Efficient Convolution Kernels for Dependency and Constituent Syntactic Trees , 2006, ECML.

[5]  Walter Daelemans,et al.  On the Limits of Sentence Compression by Deletion , 2010, Empirical Methods in Natural Language Generation.

[6]  Daniel Marcu,et al.  Summarization beyond sentence extraction: A probabilistic approach to sentence compression , 2002, Artif. Intell..

[7]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[8]  Kevin Gimpel,et al.  Towards Universal Paraphrastic Sentence Embeddings , 2015, ICLR.

[9]  Li Zhao,et al.  Attention-based LSTM for Aspect-level Sentiment Classification , 2016, EMNLP.

[10]  Sanjeev Arora,et al.  A Simple but Tough-to-Beat Baseline for Sentence Embeddings , 2017, ICLR.

[11]  Christopher D. Manning,et al.  Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks , 2015, ACL.

[12]  M. de Rijke,et al.  Siamese CBOW: Optimizing Word Embeddings for Sentence Representations , 2016, ACL.

[13]  Jürgen Schmidhuber,et al.  Learning Precise Timing with LSTM Recurrent Networks , 2003, J. Mach. Learn. Res..

[14]  Mirella Lapata,et al.  Large Margin Synchronous Generation and its Application to Sentence Compression , 2007, EMNLP.

[15]  Hal Daumé,et al.  Deep Unordered Composition Rivals Syntactic Methods for Text Classification , 2015, ACL.

[16]  Roberto Navigli,et al.  From senses to texts: An all-in-one graph-based approach for measuring semantic similarity , 2015, Artif. Intell..

[17]  Yaohui Jin,et al.  A Generalized Recurrent Neural Architecture for Text Classification with Multi-Task Learning , 2017, IJCAI.

[18]  Holger Schwenk,et al.  Supervised Learning of Universal Sentence Representations from Natural Language Inference Data , 2017, EMNLP.

[19]  Jie Wu,et al.  Active opinion-formation in online social networks , 2017, IEEE INFOCOM 2017 - IEEE Conference on Computer Communications.

[20]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[21]  Yoshua Bengio,et al.  A Neural Probabilistic Language Model , 2003, J. Mach. Learn. Res..

[22]  Jiajun Zhang,et al.  Learning Sentence Representation with Guidance of Human Attention , 2016, IJCAI.

[23]  Mark Dredze,et al.  Learning Composition Models for Phrase Embeddings , 2015, TACL.

[24]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[25]  Roberto Basili,et al.  Semantic convolution kernels over dependency trees: smoothed partial tree kernel , 2011, CIKM '11.

[26]  Quoc V. Le,et al.  Distributed Representations of Sentences and Documents , 2014, ICML.

[27]  Jie Wu,et al.  On Selecting Recommenders for Trust Evaluation in Online Social Networks , 2015, ACM Trans. Internet Techn..

[28]  Kevin Gimpel,et al.  From Paraphrase Database to Compositional Paraphrase Model and Back , 2015, Transactions of the Association for Computational Linguistics.

[29]  Zhe Gan,et al.  Learning Generic Sentence Representations Using Convolutional Neural Networks , 2016, EMNLP.

[30]  Sanja Fidler,et al.  Skip-Thought Vectors , 2015, NIPS.

[31]  Roberto Basili,et al.  Structured Lexical Similarity via Convolution Kernels on Dependency Trees , 2011, EMNLP.

[32]  Ion Androutsopoulos,et al.  A New Sentence Compression Dataset and Its Use in an Abstractive Generate-and-Rank Sentence Compressor , 2011 .

[33]  Bin Yao,et al.  ACV-tree: A New Method for Sentence Similarity Modeling , 2018, IJCAI.

[34]  A. Carnie.  Syntax: A Generative Introduction , 2002 .