An Application of Operational Research to Computational Linguistics: Word Ambiguity

Abstract This paper draws on graph theory and optimization techniques to develop a new measure of word ambiguity (e.g., homonymy and polysemy) for use in psycholinguistic research. This measure provides information regarding the uncertainty of the intended meaning of English words. Specifically, data about fifty thousand distinct words was collected from a corpus of close to six hundred million words. These data are used to generate information about word association which forms a basis for the creation of semantic graphs from which clusters are created and analyzed. The clusters identify groups of words related to the different meanings of a word and are used to calculate a set of relative probabilities for the meanings. These are in turn used to calculate the information entropy for the word, which acts as a surrogate measure of ambiguity. A genetic algorithm is used to optimally determine parameters for our formula for word association and for the graph clustering algorithm. The effectiveness of this application is demonstrated with examples from psycholinguistic research.

[1]  Jennifer M Rodd.  The effect of semantic ambiguity on reading aloud: A twist in the tale , 2004, Psychonomic bulletin & review.

[2]  Ekaterini Klepousniotou.  The Processing of Lexical Ambiguity: Homonymy and Polysemy in the Mental Lexicon , 2002, Brain and Language.

[3]  Hinrich Schütze,et al.  Automatic Word Sense Discrimination , 1998, Comput. Linguistics.

[4]  Dominic Widdows,et al.  A Graph Model for Unsupervised Lexical Acquisition , 2002, COLING.

[5]  Michael N Jones,et al.  Representing word meaning and order information in a composite holographic lexicon. , 2007, Psychological review.

[6]  David Poeppel,et al.  The effects of homonymy and polysemy on lexical access: an MEG study. , 2005, Brain research. Cognitive brain research.

[7]  Patrick Pantel,et al.  Discovering word senses from text , 2002, KDD.

[8]  M. Gernsbacher.  Resolving 20 years of inconsistent interactions between lexical familiarity and orthography, concreteness, and polysemy. , 1984, Journal of experimental psychology. General.

[9]  G. Murphy,et al.  The Representation of Polysemous Words , 2001 .

[10]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[11]  George Bosilca,et al.  Open MPI: Goals, Concept, and Design of a Next Generation MPI Implementation , 2004, PVM/MPI.

[12]  Greg Burns,et al.  LAM: An Open Cluster Environment for MPI , 2002 .

[13]  Lori Buchanan,et al.  WINDSOR: Windsor improved norms of distance and similarity of representations of semantics , 2008, Behavior research methods.

[14]  Jon. Casey.  The mathematical structure of semantic distances in language analysis. , 2005 .

[15]  Curt Burgess,et al.  Producing high-dimensional semantic spaces from lexical co-occurrence , 1996 .

[16]  R. Shillcock,et al.  Rethinking the Word Frequency Effect: The Neglected Role of Distributional Information in Lexical Processing , 2001, Language and speech.

[17]  William D. Marslen-Wilson,et al.  Modelling the effects of semantic ambiguity in word recognition , 2004, Cogn. Sci..

[18]  David R. Karger,et al.  Scatter/Gather: a cluster-based approach to browsing large document collections , 1992, SIGIR '92.

[19]  I. D. Melamed.  Measuring Semantic Entropy , 1997 .

[20]  Dekang Lin,et al.  Automatic Retrieval and Clustering of Similar Words , 1998, ACL.

[21]  W. Marslen-Wilson,et al.  Making Sense of Semantic Ambiguity: Semantic Competition in Lexical Access , 2002 .

[22]  Tamiko Azuma,et al.  Why SAFE Is Better Than FAST: The Relatedness of a Word's Meanings Affects Lexical Decision Times , 1997 .

[23]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[24]  Lori Buchanan,et al.  Grounding co-occurrence: Identifying features in a lexical co-occurrence model of semantic memory , 2009, Behavior research methods.

[25]  Curt Burgess,et al.  Modelling Parsing Constraints with High-dimensional Context Space , 1997 .

[26]  Robert M. Haralick,et al.  Decomposition of Two-Dimensional Shapes by Graph-Theoretic Clustering , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  H. Schütze,et al.  Dimensions of meaning , 1992, Supercomputing '92.

[28]  H. Rubenstein,et al.  Homographic entries in the internal lexicon , 1970 .

[29]  C. Burgess,et al.  Semantic and associative priming in the cerebral hemispheres: Some words do, some words don't … sometimes, some places , 1990, Brain and Language.

[30]  Andrew Lumsdaine,et al.  A Component Architecture for LAM/MPI , 2003, PVM/MPI.

[31]  J. Jastrzembski.  Multiple meanings, number of related meanings, frequency of occurrence, and the lexicon , 1981, Cognitive Psychology.

[32]  Enrico Motta,et al.  Solving Semantic Ambiguity to Improve Semantic Web based Ontology Matching , 2007, OM.

[33]  Dominic Widdows,et al.  Using Curvature and Markov Clustering in Graphs for Lexical Acquisition and Word Sense Discrimination , 2004 .

[34]  Kevin Durda,et al.  Investigating the structure of semantic memory. , 2006 .

[35]  Cyrus Shaoul,et al.  Word frequency effects in high-dimensional co-occurrence models: A new approach , 2006, Behavior research methods.

[36]  S. Dongen.  A cluster algorithm for graphs , 2000 .

[37]  Douglas L. T. Rohde.  An Improved Method for Deriving Word Meaning from Lexical Co-Occurrence , 2004 .

[38]  James Kelly,et al.  AutoClass: A Bayesian Classification System , 1993, ML.

[39]  Dominic Widdows,et al.  Discovering Corpus-Specific Word Senses , 2003, EACL.

[40]  P. Dixon,et al.  University of Alberta norms of relative meaning frequency for 566 homographs , 1994, Memory & cognition.

[41]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[42]  Fred J. Damerau,et al.  Generating and Evaluating Domain-Oriented Multi-Word Terms from Texts , 1993, Inf. Process. Manag..

[43]  Patrick Pantel,et al.  Efficiently Clustering Documents with Committees , 2002, PRICAI.

[44]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[45]  Naftali Tishby,et al.  Distributional Clustering of English Words , 1993, ACL.

[46]  Patrick Pantel,et al.  Concept Discovery from Text , 2002, COLING.

[47]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .