Improving Hypernymy Extraction with Distributional Semantic Classes

In this paper, we show for the first time how distributionally-induced semantic classes can be helpful for extraction of hypernyms. We present a method for (1) inducing sense-aware semantic classes using distributional semantics and (2) using these induced semantic classes for filtering noisy hypernymy relations. Denoising of hypernyms is performed by labeling each semantic class with its hypernyms. On one hand, this allows us to filter out wrong extractions using the global structure of the distributionally similar senses. On the other hand, we infer missing hypernyms via label propagation to cluster terms. We conduct a large-scale crowdsourcing study showing that processing of automatically extracted hypernyms using our approach improves the quality of the hypernymy extraction both in terms of precision and recall. Furthermore, we show the utility of our method in the domain taxonomy induction task, achieving the state-of-the-art results on a benchmarking dataset.

[1]  Jens Lehmann,et al.  DBpedia: A Nucleus for a Web of Open Data , 2007, ISWC/ASWC.

[2]  Ido Dagan,et al.  Integrating Pattern-Based and Distributional Similarity Methods for Lexical Entailment Acquisition , 2006, ACL.

[3]  Justus J. Randolph Free-Marginal Multirater Kappa (multirater K[free]): An Alternative to Fleiss' Fixed-Marginal Multirater Kappa. , 2005 .

[4]  Gemma Boleda,et al.  Inclusive yet Selective: Supervised Distributional Hypernymy Detection , 2014, COLING.

[5]  Ekaterina Vylomova,et al.  Take and Took, Gaggle and Goose, Book and Read: Evaluating the Utility of Vector Differences for Lexical Relation Learning , 2015, ACL.

[6]  Ido Dagan,et al.  Global Learning of Typed Entailment Rules , 2011, ACL.

[7]  Anton Osokin,et al.  Breaking Sticks and Ambiguities with Adaptive Skip-gram , 2015, AISTATS.

[8]  Christian Biemann,et al.  Negative Sampling Improves Hypernymy Extraction Based on Projection Learning , 2017, EACL.

[9]  Silvia Bernardini,et al.  Introducing and evaluating ukWaC , a very large web-derived corpus of English , 2008 .

[10]  Lei Shi,et al.  Putting Pieces Together: Combining FrameNet, VerbNet and WordNet for Robust Semantic Parsing , 2005, CICLing.

[11]  Christian Biemann,et al.  Chinese Whispers - an Efficient Graph Clustering Algorithm and its Application to Natural Language Processing Problems , 2006 .

[12]  Christian Biemann Creating a system for lexical substitutions from scratch using crowdsourcing , 2013, Lang. Resour. Evaluation.

[13]  Ido Dagan,et al.  Improving Hypernymy Detection with an Integrated Path-based and Distributional Method , 2016, ACL.

[14]  Stefano Faralli,et al.  Unsupervised, Knowledge-Free, and Interpretable Word Sense Disambiguation , 2017, EMNLP.

[15]  Stefano Faralli,et al.  TAXI at SemEval-2016 Task 13: a Taxonomy Induction Method based on Lexico-Syntactic Patterns, Substrings and Focused Crawling , 2016, *SEMEVAL.

[16]  Stefano Faralli,et al.  OntoLearn Reloaded: A Graph-Based Algorithm for Taxonomy Induction , 2013, CL.

[17]  Núria Bel,et al.  Reading Between the Lines: Overcoming Data Sparsity for Accurate Classification of Lexical Relationships , 2015, *SEM@NAACL-HLT.

[18]  Christian Biemann,et al.  Ontology Learning from Text: A Survey of Methods , 2005, LDV Forum.

[19]  Tonio Wandmacher,et al.  How semantic is Latent Semantic Analysis? , 2005, JEPTALNRECITAL.

[20]  Maria Leonor Pacheco,et al.  of the Association for Computational Linguistics: , 2001 .

[21]  Fang Liu,et al.  Improving Question Retrieval in Community Question Answering Using World Knowledge , 2013, IJCAI.

[22]  Dominic Widdows,et al.  A Graph Model for Unsupervised Lexical Acquisition , 2002, COLING.

[23]  Steffen Staab,et al.  Ontology Learning from Text , 2000, NLDB.

[24]  David J. Weir,et al.  Learning to Distinguish Hypernyms and Co-Hyponyms , 2014, COLING.

[25]  Martin Everett,et al.  Ego network betweenness , 2005, Soc. Networks.

[26]  Patrick Pantel,et al.  Automatically Labeling Semantic Classes , 2004, NAACL.

[27]  Yves Peirsman,et al.  Modelling Word Similarity: an Evaluation of Automatic Synonymy Extraction Algorithms , 2008, LREC.

[28]  Chris Biemann,et al.  Exploiting the Leipzig Corpora Collection , 2006 .

[29]  Iryna Gurevych,et al.  DKPro Agreement: An Open-Source Java Library for Measuring Inter-Rater Agreement , 2014, COLING.

[30]  Patrick Pantel,et al.  Espresso: Leveraging Generic Patterns for Automatically Harvesting Semantic Relations , 2006, ACL.

[31]  Joshua B. Tenenbaum,et al.  The Large-Scale Structure of Semantic Networks: Statistical Analyses and a Model of Semantic Growth , 2001, Cogn. Sci..

[32]  Daniel Jurafsky,et al.  Learning Syntactic Patterns for Automatic Hypernym Discovery , 2004, NIPS.

[33]  Erik F. Tjong Kim Sang,et al.  Extracting Hypernym Pairs from the Web , 2007, ACL.

[34]  Chong Wang,et al.  Reading Tea Leaves: How Humans Interpret Topic Models , 2009, NIPS.

[35]  Patrick Pantel,et al.  Discovering word senses from text , 2002, KDD.

[36]  BiemannChris Creating a system for lexical substitutions from scratch using crowdsourcing , 2013 .

[37]  Stefano Faralli,et al.  A Large DataBase of Hypernymy Relations Extracted from the Web , 2016, LREC.

[38]  Christian Biemann,et al.  Text: now in 2D! A framework for lexical expansion with contextual similarity , 2013, J. Lang. Model..

[39]  Wanxiang Che,et al.  Learning Semantic Hierarchies via Word Embeddings , 2014, ACL.

[40]  Marti A. Hearst Automatic Acquisition of Hyponyms from Large Text Corpora , 1992, COLING.

[41]  Sharon A. Caraballo Automatic construction of a hypernym-labeled noun hierarchy from text , 1999, ACL.

[42]  Goran Glavas,et al.  Dual Tensor Model for Detecting Asymmetric Lexico-Semantic Relations , 2017, EMNLP.

[43]  Zornitsa Kozareva,et al.  Tailoring the automated construction of large-scale taxonomies using the web , 2013, Lang. Resour. Evaluation.

[44]  Christian Biemann,et al.  Watset: Automatic Induction of Synsets from a Graph of Synonyms , 2017, ACL.

[45]  Steffen Staab,et al.  Ontology Learning from Text , 2000, International Conference on Applications of Natural Language to Data Bases.

[46]  Dmitry Ustalov,et al.  YARN: Spinning-in-Progress , 2016, GWC.

[47]  Patrick Pantel,et al.  Induction of semantic classes from natural language text , 2001, KDD '01.

[48]  Stefano Faralli,et al.  Linked Disambiguated Distributional Semantic Networks , 2016, International Semantic Web Conference.

[49]  Stefano Faralli,et al.  A framework for enriching lexical semantic resources with distributional semantics , 2017, Natural Language Engineering.

[50]  Omer Levy,et al.  Do Supervised Distributional Methods Really Learn Lexical Inference Relations? , 2015, NAACL.

[51]  Tomaz Erjavec,et al.  The JRC-Acquis: A Multilingual Aligned Parallel Corpus with 20+ Languages , 2006, LREC.

[52]  Ewan Klein,et al.  Natural Language Processing with Python , 2009 .

[53]  Paul Buitelaar,et al.  SemEval-2015 Task 17: Taxonomy Extraction Evaluation (TExEval) , 2015, SemEval@NAACL-HLT.

[54]  Simone Paolo Ponzetto,et al.  BabelNet: The automatic construction, evaluation and application of a wide-coverage multilingual semantic network , 2012, Artif. Intell..

[55]  Walter Daelemans,et al.  Pattern for Python , 2012, J. Mach. Learn. Res..

[56]  Zhiguo Gong,et al.  Web Query Expansion by WordNet , 2005, DEXA.

[57]  Andrew McCallum,et al.  Efficient Non-parametric Estimation of Multiple Embeddings per Word in Vector Space , 2014, EMNLP.