Sentence Modeling via Graph Construction and Graph Neural Networks for Semantic Textual Similarity

Recently, using graph neural networks to model the hidden features of natural language has achieved success. In this paper, a novel sentence modeling method named TextSimGNN based on graphical representation is proposed to measure the semantic textual similarity. For embedding sentences into a graphical structure, we first construct a semantic textual graph which combines textual structure information and semantic information together. Then an end-to-end graph neural network is used to measure the similarity between graph pairs. The experiments show that our method has achieved good performance in semantic textual similarity task, which proves the advantage and effectiveness of graphical representation on natural language sentence modeling.

[1]  Houfeng Wang,et al.  Text Level Graph Neural Network for Text Classification , 2019, EMNLP.

[2]  Jianxin Li,et al.  Large-Scale Hierarchical Text Classification with Recursively Regularized Deep Graph-CNN , 2018, WWW.

[3]  Tao Gui,et al.  A Lexicon-Based Graph Neural Network for Chinese NER , 2019, EMNLP.

[4]  Rabab Kreidieh Ward,et al.  Deep Sentence Embedding Using Long Short-Term Memory Networks: Analysis and Application to Information Retrieval , 2015, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[5]  Yuan Luo,et al.  Graph Convolutional Networks for Text Classification , 2018, AAAI.

[6]  Poonam Bansal,et al.  Latent Semantic Analysis: An Approach to Understand Semantic of Text , 2017, 2017 International Conference on Current Trends in Computer, Electrical, Electronics and Communication (CTCEEC).

[7]  Xianglong Liu,et al.  Graph Convolutional Network Hashing for Cross-Modal Retrieval , 2019, IJCAI.

[8]  Larry P. Heck,et al.  Learning deep structured semantic models for web search using clickthrough data , 2013, CIKM.

[9]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[10]  S. Srinivasan,et al.  Text processing in information retrieval system using vector space model , 2014, International Conference on Information Communication and Embedded Systems (ICICES2014).

[11]  Dilek Z. Hakkani-Tür,et al.  Zero-shot learning of intent embeddings for expansion by convolutional deep structured semantic models , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[12]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[13]  Yizhou Sun,et al.  SimGNN: A Neural Network Approach to Fast Graph Similarity Computation , 2018, WSDM.

[14]  Fu-Lai Chung,et al.  Deep Network Embedding for Graph Representation Learning in Signed Networks , 2019, IEEE Transactions on Cybernetics.

[15]  Yang Shao,et al.  HCTI at SemEval-2017 Task 1: Use convolutional neural network to evaluate Semantic Textual Similarity , 2017, SemEval@ACL.