Automatically acquiring a semantic network of related concepts

We describe the automatic construction of a semantic network1, in which over 3000 of the most frequently occurring monosemous nouns2 in Wikipedia (each appearing between 1,500 and 100,000 times) are linked to their semantically related concepts in the WordNet noun ontology. Relatedness between nouns is discovered automatically from co-occurrence in Wikipedia texts using an information theoretic inspired measure. Our algorithm then capitalizes on salient sense clustering among related nouns to automatically disambiguate them to their appropriate senses (i.e., concepts). Through the act of disambiguation, we begin to accumulate relatedness data for concepts denoted by polysemous nouns, as well. The resultant concept-to-concept associations, covering 17,543 nouns, and 27,312 distinct senses among them, constitute a large-scale semantic network of related concepts that can be conceived of as augmenting the WordNet noun ontology with related-to links.

[1]  Christiane Fellbaum,et al.  Lexical Chains as Representations of Context for the Detection and Correction of Malapropisms , 1998 .

[2]  D. Spence,et al.  Lexical co-occurrence and association strength , 1990 .

[3]  Antonio Torralba,et al.  Context-based vision system for place and object recognition , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[4]  Roberto Navigli,et al.  SemEval-2007 Task 07: Coarse-Grained English All-Words Task , 2007, Fourth International Workshop on Semantic Evaluations (SemEval-2007).

[5]  Roberto Navigli,et al.  Semi-Automatic Extension of Large-Scale Linguistic Knowledge Bases , 2005, FLAIRS.

[6]  Douglas B. Lenat,et al.  CYC: a large-scale investment in knowledge infrastructure , 1995, CACM.

[7]  Ari Rappoport,et al.  Efficient Unsupervised Discovery of Word Categories Using Symmetric Patterns and High Frequency Words , 2006, ACL.

[8]  Dan Moldovan,et al.  Models for the Semantic Classification of Noun Phrases , 2004, HLT-NAACL 2004.

[9]  Eneko Agirre,et al.  Publicly Available Topic Signatures for all WordNet Nominal Senses , 2004, LREC.

[10]  J. Fodor,et al.  The structure of a semantic theory , 1963 .

[11]  Patrick Pantel,et al.  Discovery of inference rules for question-answering , 2001, Natural Language Engineering.

[12]  Michael D. Lee,et al.  An Empirical Evaluation of Models of Text Document Similarity , 2005 .

[13]  William E. Moen,et al.  Using Encyclopedic Knowledge for Automatic Topic Identification , 2009, CoNLL.

[14]  Magnus Sahlgren,et al.  The Distributional Hypothesis , 2008 .

[15]  Gerhard Weikum,et al.  Deriving a Web-Scale Common Sense Fact Database , 2011, AAAI.

[16]  Donald Hindle,et al.  Noun Classification From Predicate-Argument Structures , 1990, ACL.

[17]  Montse Cuadros,et al.  KnowNet: Building a Large Net of Knowledge from the Web , 2008, COLING.

[18]  Eleanor Rosch,et al.  Principles of Categorization , 1978 .

[19]  Fernando Gomez,et al.  Evaluating a Semantic Network Automatically Constructed from Lexical Co-occurrence on a Word Sense Disambiguation Task , 2011, CoNLL.

[20]  Rada Mihalcea,et al.  Using Wikipedia for Automatic Word Sense Disambiguation , 2007, NAACL.

[21]  Evgeniy Gabrilovich,et al.  Computing Semantic Relatedness Using Wikipedia-based Explicit Semantic Analysis , 2007, IJCAI.

[22]  Eneko Agirre,et al.  A Study on Similarity and Relatedness Using Distributional and WordNet-based Approaches , 2009, NAACL.

[23]  Hugo Liu,et al.  Commonsense Reasoning in and Over Natural Language , 2004, KES.

[24]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[25]  James J. Jenkins,et al.  Word Association Norms: Grade School Through College , 1964 .

[26]  Paul M. B. Vitányi,et al.  The Google Similarity Distance , 2004, IEEE Transactions on Knowledge and Data Engineering.

[27]  Peter D. Turney.  Similarity of Semantic Relations , 2006, CL.

[28]  Gerhard Weikum,et al.  YAGO: A Large Ontology from Wikipedia and WordNet , 2008, J. Web Semant..

[29]  Ted Pedersen,et al.  Extended Gloss Overlaps as a Measure of Semantic Relatedness , 2003, IJCAI.

[30]  John B. Lowe,et al.  The Berkeley FrameNet Project , 1998, ACL.

[31]  H. Chertkow,et al.  Semantic memory , 2002, Current neurology and neuroscience reports.

[32]  Julie Elizabeth Weeds,et al.  Measures and applications of lexical distributional similarity , 2003 .

[33]  John McCarthy,et al.  Programs with common sense , 1960 .

[34]  Simone Paolo Ponzetto,et al.  WikiRelate! Computing Semantic Relatedness Using Wikipedia , 2006, AAAI.

[35]  Philip Resnik,et al.  Selectional Preference and Sense Disambiguation , 1997 .

[36]  Catherine Havasi,et al.  ConceptNet 3 : a Flexible , Multilingual Semantic Network for Common Sense Knowledge , 2007 .

[37]  G. Miller,et al.  Contextual correlates of semantic similarity , 1991 .

[38]  Eugene Charniak,et al.  Passing Markers: A Theory of Contextual Influence in Language Comprehension , 1983, Cogn. Sci..

[39]  Beth Levin,et al.  English Verb Classes and Alternations: A Preliminary Investigation , 1993 .

[40]  Mirella Lapata,et al.  An Experimental Study of Graph Connectivity for Unsupervised Word Sense Disambiguation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  David W. Conrath,et al.  Semantic Similarity Based on Corpus Statistics and Lexical Taxonomy , 1997, ROCLING/IJCLCLP.

[42]  Gilad Mishne,et al.  Using Wikipedia at the TREC QA Track , 2004, TREC.

[43]  Jordan B. Pollack,et al.  Massively Parallel Parsing: A Strongly Interactive Model of Natural Language Interpretation , 1988, Cogn. Sci..

[44]  Philip Resnik,et al.  Semantic Similarity in a Taxonomy: An Information-Based Measure and its Application to Problems of Ambiguity in Natural Language , 1999, J. Artif. Intell. Res..

[45]  Peter D. Turney.  Expressing Implicit Semantic Relations without Supervision , 2006, ACL.

[46]  David J. Weir,et al.  Co-occurrence Retrieval: A Flexible Framework for Lexical Distributional Similarity , 2005, CL.

[47]  Simone Paolo Ponzetto,et al.  Knowledge-Rich Word Sense Disambiguation Rivaling Supervised Systems , 2010, ACL.

[48]  Marti A. Hearst.  Automatic Acquisition of Hyponyms from Large Text Corpora , 1992, COLING.

[49]  Fernando Gomez,et al.  An Algorithm for Aspects of Semantic Interpretation Using an Enhanced WordNet , 2001, NAACL.

[50]  Kenneth Ward Church,et al.  Word Association Norms, Mutual Information, and Lexicography , 1989, ACL.

[51]  Fernando Gomez.  Building Verb Predicates: A Computational View , 2004, ACL.

[52]  Ian H. Witten,et al.  An effective, low-cost measure of semantic relatedness obtained from Wikipedia links , 2008 .

[53]  Jens Lehmann,et al.  DBpedia - A crystallization point for the Web of Data , 2009, J. Web Semant..

[54]  C. Fellbaum.  An Electronic Lexical Database , 1998 .

[55]  Push Singh,et al.  The Public Acquisition of Commonsense Knowledge , 2002 .

[56]  Razvan C. Bunescu,et al.  Using Encyclopedic Knowledge for Named entity Disambiguation , 2006, EACL.

[57]  Douglas B. Lenat,et al.  Mapping Ontologies into Cyc , 2002 .

[58]  Aaron Sloman,et al.  The St. Thomas Common Sense Symposium: Designing Architectures for Human-Level Intelligence , 2004, AI Mag..

[59]  Eduard H. Hovy,et al.  Learning surface text patterns for a Question Answering System , 2002, ACL.

[60]  Zellig S. Harris,et al.  Distributional Structure , 1954 .

[61]  Daniel Gildea,et al.  Automatic Labeling of Semantic Roles , 2000, ACL.

[62]  Roy Rada,et al.  Development and application of a metric on semantic nets , 1989, IEEE Trans. Syst. Man Cybern..

[63]  Ted Dunning,et al.  Accurate Methods for the Statistics of Surprise and Coincidence , 1993, CL.

[64]  Reinhard Rapp,et al.  Computation of Word Associations Based on Co-occurrences of Words in Large Corpora , 1993, VLC@ACL.

[65]  Doug Downey,et al.  Web-scale information extraction in knowitall: (preliminary results) , 2004, WWW '04.

[66]  Ellen Riloff,et al.  Learning Dictionaries for Information Extraction by Multi-Level Bootstrapping , 1999, AAAI/IAAI.

[67]  Huaiyu Zhu.  On Information and Sufficiency , 1997 .

[68]  David Yarowsky,et al.  One Sense Per Discourse , 1992, HLT.

[69]  Patrick Pantel,et al.  Espresso: Leveraging Generic Patterns for Automatically Harvesting Semantic Relations , 2006, ACL.

[70]  Om P. Damani,et al.  Lexical Co-occurrence, Statistical Significance, and Word Association , 2011, EMNLP.

[71]  Stan Szpakowicz,et al.  Roget's thesaurus and semantic similarity , 2012, RANLP.

[72]  Ted Pedersen,et al.  WordNet::Similarity - Measuring the Relatedness of Concepts , 2004, NAACL.

[73]  Simone Paolo Ponzetto,et al.  Large-Scale Taxonomy Mapping for Restructuring and Integrating Wikipedia , 2009, IJCAI.

[74]  Erik T. Mueller,et al.  Open Mind Common Sense: Knowledge Acquisition from the General Public , 2002, OTM.

[75]  Eric Brill,et al.  Transformation-Based Error-Driven Learning and Natural Language Processing: A Case Study in Part-of-Speech Tagging , 1995, CL.

[76]  Ian H. Witten,et al.  Learning to link with wikipedia , 2008, CIKM '08.

[77]  Dan I. Moldovan,et al.  Automatic Discovery of Part-Whole Relations , 2006, CL.

[78]  R. Ratcliff,et al.  Spreading activation versus compound cue accounts of priming: mediated priming revisited. , 1992, Journal of experimental psychology. Learning, memory, and cognition.

[79]  John B. Goodenough,et al.  Contextual correlates of synonymy , 1965, CACM.

[80]  Ted Pedersen,et al.  Using WordNet-based Context Vectors to Estimate the Semantic Relatedness of Concepts , 2006 .

[81]  Olena Medelyan,et al.  Integrating Cyc and Wikipedia: Folksonomy meets rigorously defined common-sense , 2008, AAAI 2008.

[82]  Eugene Charniak,et al.  Finding Parts in Very Large Corpora , 1999, ACL.

[83]  Patrick Pantel,et al.  VerbOcean: Mining the Web for Fine-Grained Semantic Verb Relations , 2004, EMNLP.

[84]  Giuseppe Attardi,et al.  Ranking very many typed entities on wikipedia , 2007, CIKM '07.

[85]  Praveen Paritosh,et al.  Freebase: a collaboratively created graph database for structuring human knowledge , 2008, SIGMOD Conference.

[86]  Gregory Grefenstette,et al.  Explorations in automatic thesaurus discovery , 1994 .

[87]  Rada Mihalcea,et al.  Wikify!: linking documents to encyclopedic knowledge , 2007, CIKM '07.

[88]  Roy Rada,et al.  Ranking documents with a thesaurus , 1989, JASIS.

[89]  David Yarowsky,et al.  One Sense per Collocation , 1993, HLT.

[90]  Hugo Liu,et al.  ConceptNet — A Practical Commonsense Reasoning Tool-Kit , 2004 .

[91]  Philip Resnik,et al.  Using Information Content to Evaluate Semantic Similarity in a Taxonomy , 1995, IJCAI.

[92]  James R. Curran,et al.  Scaling Distributional Similarity to Large Corpora , 2006, ACL.

[93]  Christiane Fellbaum,et al.  Combining Local Context and Wordnet Similarity for Word Sense Identification , 1998 .

[94]  Fernando Gomez,et al.  Extracting Ontological Selectional Preferences for Non-Pertainym Adjectives from the Google Corpus , 2010, AAAI.

[95]  C. Fillmore.  FRAME SEMANTICS AND THE NATURE OF LANGUAGE * , 1976 .

[96]  Allan Collins,et al.  How to make a language user. , 1972 .

[97]  Michael E. Lesk,et al.  Automatic sense disambiguation using machine readable dictionaries: how to tell a pine cone from an ice cream cone , 1986, SIGDOC '86.

[98]  Christiane Fellbaum,et al.  Nouns in WordNet , 1998 .

[99]  Oren Etzioni,et al.  Identifying Relations for Open Information Extraction , 2011, EMNLP.

[100]  K. Bollacker,et al.  A Platform for Scalable, Collaborative, Structured Information Integration , 2007 .

[101]  Estevam R. Hruschka,et al.  Toward an Architecture for Never-Ending Language Learning , 2010, AAAI.

[102]  Ming Zhou,et al.  Identifying Synonyms among Distributionally Similar Words , 2003, IJCAI.

[103]  Dekang Lin,et al.  An Information-Theoretic Definition of Similarity , 1998, ICML.

[104]  Thad Hughes,et al.  Lexical Semantic Relatedness with Random Graph Walks , 2007, EMNLP.

[105]  Allan Collins,et al.  A spreading-activation theory of semantic processing , 1975 .

[106]  M. Ross Quillian,et al.  The teachable language comprehender: a simulation program and theory of language , 1969, CACM.

[107]  Graeme Hirst,et al.  Evaluating WordNet-based Measures of Lexical Semantic Relatedness , 2006, CL.