Distantly-Supervised Long-Tailed Relation Extraction Using Constraint Graphs

Label noise and long-tailed distributions are two major challenges in distantly supervised relation extraction. Recent studies have shown great progress on denoising, but pay little attention to the problem of long-tailed relations. In this paper, we introduce constraint graphs to model the dependencies between relation labels. On top of that, we further propose a novel constraint graph-based relation extraction framework(CGRE) to handle the two challenges simultaneously. CGRE employs graph convolution networks (GCNs) to propagate information from data-rich relation nodes to data-poor relation nodes, and thus boosts the representation learning of long-tailed relations. To further improve the noise immunity, a constraint-aware attention module is designed in CGRE to integrate the constraint information. Experimental results on a widely-used benchmark dataset indicate that our approach achieves significant improvements over the previous methods for both denoising and long-tailed relation extraction. Our dataset and codes are available at https://github.com/tmliang/CGRE.

[1]  Li Zhao,et al.  Reinforcement Learning for Relation Classification From Noisy Data , 2018, AAAI.

[2]  Ning Ding,et al.  Modeling Relation Paths for Knowledge Graph Completion , 2020, IEEE Transactions on Knowledge and Data Engineering.

[3]  Danqi Chen,et al.  Position-aware Attention and Supervised Data Improve Slot Filling , 2017, EMNLP.

[4]  Ming Gao,et al.  Improving Neural Relation Extraction with Implicit Mutual Relations , 2019, 2020 IEEE 36th International Conference on Data Engineering (ICDE).

[5]  Gregory Gutin,et al.  Digraphs - theory, algorithms and applications , 2002 .

[6]  Han Xiao,et al.  Knowledge Graph Embedding Based on Multi-View Clustering Framework , 2019, IEEE Transactions on Knowledge and Data Engineering.

[7]  Hans Uszkoreit,et al.  Large-Scale Learning of Relation-Extraction Rules with Distant Supervision from the Web , 2012, International Semantic Web Conference.

[8]  Jun Zhao,et al.  Distant Supervision for Relation Extraction with Sentence-Level Attention and Entity Descriptions , 2017, AAAI.

[9]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[10]  Dmitri V. Kalashnikov,et al.  Web People Search via Connection Analysis , 2008, IEEE Transactions on Knowledge and Data Engineering.

[11]  Xinyan Xiao,et al.  ARNOR: Attention Regularization based Noise Reduction for Distant Supervision Relation Classification , 2019, ACL.

[12]  Maosong Sun,et al.  Learning from Context or Names? An Empirical Study on Neural Relation Extraction , 2020, EMNLP.

[13]  Luke S. Zettlemoyer,et al.  Knowledge-Based Weak Supervision for Information Extraction of Overlapping Relations , 2011, ACL.

[14]  Maosong Sun,et al.  FewRel 2.0: Towards More Challenging Few-Shot Relation Classification , 2019, EMNLP.

[15]  Xiao Liu,et al.  Distant Supervision for Relation Extraction with Linear Attenuation Simulation and Non-IID Relevance Embedding , 2019, AAAI.

[16]  Yang Liu,et al.  Exploring Fine-grained Entity Type Constraints for Distantly Supervised Relation Extraction , 2014, COLING.

[17]  Chiranjib Bhattacharyya,et al.  RESIDE: Improving Distantly-Supervised Neural Relation Extraction using Side Information , 2018, EMNLP.

[18]  Man Zhu,et al.  Exploring Long Tail Data in Distantly Supervised Relation Extraction , 2016, NLPCC/ICCPOL.

[19]  Xi Chen,et al.  Long-tail Relation Extraction via Knowledge Graph Embeddings and Graph Convolution Networks , 2019, NAACL.

[20]  Huifang Ma,et al.  Adaptive Graph Convolutional Networks with Attention Mechanism for Relation Extraction , 2020, 2020 International Joint Conference on Neural Networks (IJCNN).

[21]  Linmei Hu,et al.  Improving Distantly-Supervised Relation Extraction with Joint Label Embedding , 2019, EMNLP.

[22]  Gholamreza Haffari,et al.  Few-shot Complex Knowledge Base Question Answering via Meta Reinforcement Learning , 2020, EMNLP.

[23]  Roland Vollgraf,et al.  FLAIR: An Easy-to-Use Framework for State-of-the-Art NLP , 2019, NAACL.

[24]  Hwee Tou Ng,et al.  Towards Robust Linguistic Analysis using OntoNotes , 2013, CoNLL.

[25]  Daniel Jurafsky,et al.  Distant supervision for relation extraction without labeled data , 2009, ACL.

[26]  William Yang Wang,et al.  DSGAN: Generative Adversarial Training for Distant Supervision Relation Extraction , 2018, ACL.

[27]  Wei Zhang,et al.  Label-Free Distant Supervision for Relation Extraction via Knowledge Graph Embedding , 2018, EMNLP.

[28]  Kai Fan,et al.  Improving Distantly Supervised Relation Extraction with Neural Noise Converter and Conditional Optimal Selector , 2018, AAAI.

[29]  Jun Zhao,et al.  Distant Supervision for Relation Extraction via Piecewise Convolutional Neural Networks , 2015, EMNLP.

[30]  Zhiyuan Liu,et al.  Neural Relation Extraction with Selective Attention over Instances , 2016, ACL.

[31]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[32]  Tao Shen,et al.  Self-Attention Enhanced Selective Gate with Entity-Aware Embedding for Distantly Supervised Relation Extraction , 2019, AAAI.

[33]  Zhiyuan Liu,et al.  Hierarchical Relation Extraction with Coarse-to-Fine Grained Attention , 2018, EMNLP.

[34]  Yuming Shang,et al.  Are Noisy Sentences Useless for Distant Supervised Relation Extraction? , 2019, AAAI.

[35]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[36]  Ji-Rong Wen,et al.  Generating Query Facets Using Knowledge Bases , 2017, IEEE Transactions on Knowledge and Data Engineering.

[37]  Diyah Puspitaningrum.  Improving Performance of Relation Extraction Algorithm via Leveled Adversarial PCNN and Database Expansion , 2019, 2019 7th International Conference on Cyber and IT Service Management (CITSM).

[38]  Wei Fan,et al.  Cooperative Denoising for Distantly Supervised Relation Extraction , 2018, COLING.

[39]  Ramesh Nallapati,et al.  Multi-instance Multi-label Learning for Relation Extraction , 2012, EMNLP.

[40]  Qing Zhang,et al.  Noise-Clustered Distant Supervision for Relation Extraction: A Nonparametric Bayesian Perspective , 2017, EMNLP.

[41]  Leonhard Hennig,et al.  Fine-tuning Pre-Trained Transformer Language Models to Distantly Supervised Relation Extraction , 2019, ACL.

[42]  Andrew McCallum,et al.  Modeling Relations and Their Mentions without Labeled Text , 2010, ECML/PKDD.

[43]  Aoying Zhou,et al.  Learning Relation Prototype From Unlabeled Texts for Long-Tail Relation Extraction , 2020, IEEE Transactions on Knowledge and Data Engineering.

[44]  Preslav Nakov,et al.  SemEval-2010 Task 8: Multi-Way Classification of Semantic Relations Between Pairs of Nominals , 2009, SEW@NAACL-HLT.

[45]  Danqi Chen,et al.  A Frustratingly Easy Approach for Joint Entity and Relation Extraction , 2020, ArXiv.

[46]  David Bamman,et al.  Adversarial Training for Relation Extraction , 2017, EMNLP.

[47]  Lei Zou,et al.  Answering Natural Language Questions by Subgraph Matching over Knowledge Graphs , 2018, IEEE Transactions on Knowledge and Data Engineering.

[48]  Zhiyuan Liu,et al.  FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation , 2018, EMNLP.

[49]  Zhen-Hua Ling,et al.  Distant Supervision Relation Extraction with Intra-Bag and Inter-Bag Attentions , 2019, NAACL.

[50]  Hai Zhao,et al.  GAN Driven Semi-distant Supervision for Relation Extraction , 2019, NAACL.

[51]  Diego Marcheggiani,et al.  Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling , 2017, EMNLP.