Matching Article Pairs with Graphical Decomposition and Convolutions

Identifying the relationship between two articles, e.g., whether two articles published from different sources describe the same breaking news, is critical to many document understanding tasks. Existing approaches for modeling and matching sentence pairs do not perform well in matching longer documents, which embody more complex interactions between the enclosed entities than a sentence does. To model article pairs, we propose the Concept Interaction Graph to represent an article as a graph of concepts. We then match a pair of articles by comparing the sentences that enclose the same concept vertex through a series of encoding techniques, and aggregate the matching signals through a graph convolutional network. To facilitate the evaluation of long article matching, we have created two datasets, each consisting of about 30K pairs of breaking news articles covering diverse topics in the open domain. Extensive evaluations of the proposed methods on the two datasets demonstrate significant improvements over a wide range of state-of-the-art methods for natural language matching.

[1]  Takenobu Tokunaga,et al.  Evaluating text coherence based on semantic similarity graph , 2017, TextGraphs@ACL.

[2]  Yu Xu,et al.  Multiresolution Graph Attention Networks for Relevance Matching , 2018, CIKM.

[3]  Xueqi Cheng,et al.  Text Matching as Image Recognition , 2016, AAAI.

[4]  Collin F. Baker,et al.  Graph Methods for Multilingual FrameNets , 2017, TextGraphs@ACL.

[5]  Michalis Vazirgiannis,et al.  Graph-of-word and TW-IDF: new approach to ad hoc IR , 2013, CIKM.

[6]  Abraham Kandel,et al.  Clustering of Web Documents using a Graph Model , 2003, Web Document Analysis.

[7]  Alessandro Moschitti,et al.  Learning to Rank Short Text Pairs with Convolutional Deep Neural Networks , 2015, SIGIR.

[8]  Jonas Mueller,et al.  Siamese Recurrent Architectures for Learning Sentence Similarity , 2016, AAAI.

[9]  Dafna Shahaf,et al.  Information cartography: creating zoomable, large-scale maps of information , 2013, KDD.

[10]  Zhoujun Li,et al.  Knowledge Enhanced Hybrid Neural Network for Text Matching , 2018, AAAI.

[11]  Mihai Surdeanu,et al.  The Stanford CoreNLP Natural Language Processing Toolkit , 2014, ACL.

[12]  Yu Xu,et al.  Growing Story Forest Online from Massive Breaking News , 2017, CIKM.

[13]  Heeyoung Lee,et al.  Joint Entity and Event Coreference Resolution across Documents , 2012, EMNLP.

[14]  Jure Leskovec,et al.  Representation Learning on Graphs: Methods and Applications , 2017, IEEE Data Eng. Bull..

[15]  Sanda M. Harabagiu,et al.  Learning Textual Graph Patterns to Detect Causal Event Relations , 2010, FLAIRS.

[16]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[17]  Heeyoung Lee,et al.  Deterministic Coreference Resolution Based on Entity-Centric, Precision-Ranked Rules , 2013, CL.

[18]  Cheng Li,et al.  Semantic Text Matching for Long-Form Documents , 2019, WWW.

[19]  Hugo Zaragoza,et al.  The Probabilistic Relevance Framework: BM25 and Beyond , 2009, Found. Trends Inf. Retr..

[20]  Louiqa Raschid,et al.  A Graph Analytical Approach for Topic Detection , 2013, TOIT.

[21]  Tommaso Caselli,et al.  Storylines for structuring massive streams of news , 2015 .

[22]  Craig A. Knoblock,et al.  Efficient Graph-Based Document Similarity , 2016, ESWC.

[23]  Hang Li,et al.  Convolutional Neural Network Architectures for Matching Natural Language Sentences , 2014, NIPS.

[24]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[25]  Nick Craswell,et al.  Learning to Match using Local and Distributed Representations of Text for Web Search , 2016, WWW.

[26]  Xueqi Cheng,et al.  A Deep Architecture for Semantic Matching with Multiple Positional Sentence Representations , 2015, AAAI.

[27]  Xueqi Cheng,et al.  MatchZoo: A Toolkit for Deep Text Matching , 2017, ArXiv.

[28]  Rajeev Motwani,et al.  The PageRank Citation Ranking : Bringing Order to the Web , 1999, WWW 1999.

[29]  Yannis Stavrakas,et al.  Shortest-Path Graph Kernels for Document Similarity , 2017, EMNLP.

[30]  Piek T. J. M. Vossen,et al.  SemEval-2018 Task 5: Counting Events and Participants in the Long Tail , 2018, *SEMEVAL.

[31]  Stefan Selzer,et al.  Storyline detection and tracking using Dynamic Latent Dirichlet Allocation , 2016, Proceedings of the 2nd Workshop on Computing News Storylines (CNS 2016).

[32]  Marko Grobelnik,et al.  Learning Sub-structures of Document Semantic Graphs for Document Summarization , 2004 .

[33]  Xuanjing Huang,et al.  Convolutional Neural Tensor Network Architecture for Community-Based Question Answering , 2015, IJCAI.

[34]  Rada Mihalcea,et al.  TextRank: Bringing Order into Text , 2004, EMNLP.

[35]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[36]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[37]  Larry P. Heck,et al.  Learning deep structured semantic models for web search using clickthrough data , 2013, CIKM.

[38]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[39]  Sanda M. Harabagiu,et al.  Unsupervised Event Coreference Resolution with Rich Linguistic Features , 2010, ACL.

[40]  Deyu Zhou,et al.  An Unsupervised Bayesian Modelling Approach for Storyline Detection on News Articles , 2015, EMNLP.

[41]  Maarten Versteegh,et al.  Learning Text Similarity with Siamese Recurrent Networks , 2016, Rep4NLP@ACL.

[42]  Yelong Shen,et al.  Learning semantic representations using convolutional neural networks for web search , 2014, WWW.

[43]  Yu Xu,et al.  Matching Natural Language Sentences with Hierarchical Sentence Factorization , 2018, WWW.

[44]  Jens Lehmann,et al.  DBpedia: A Nucleus for a Web of Open Data , 2007, ISWC/ASWC.