Word Sense Disambiguation within a Multilingual Framework

Word Sense Disambiguation (WSD) is the process of resolving the meaning of a word unambiguously in a given natural language context. Within the scope of this thesis, it is the process of marking text with explicit sense labels. What constitutes a sense is a subject of great debate. An appealing perspective, aims to define senses in terms of their multilingual correspondences, an idea explored by several researchers, Dyvik (1998), Ide (1999), Resnik & Yarowsky (1999), and Chugur, Gonzalo & Verdejo (2002) but to date it has not been given any practical demonstration. This thesis is an empirical validation of these ideas of characterizing word meaning using cross-linguistic correspondences. The idea is that word meaning or word sense is quantifiable as much as it is uniquely translated in some language or set of languages. Consequently, we address the problem of WSD from a multilingual perspective; we expand the notion of context to encompass multilingual evidence. We devise a new approach to resolve word sense ambiguity in natural language, using a source of information that was never exploited on a large scale for WSD before. The core of the work presented builds on exploiting word correspondences across languages for sense distinction. In essence, it is a practical and functional implementation of a basic idea common to research interest in defining word meanings in cross-linguistic terms. We devise an algorithm, SALAAM for Sense Assignment Leveraging Alignment And Multilinguality, that empirically investigates the feasibility and the validity of utilizing translations for WSD. SALAAM is an unsupervised approach for word sense tagging of large amounts of text given a parallel corpus—texts in translation—and a sense inventory for one of the languages in the corpus. Using SALAAM, we obtain large amounts of sense annotated data in both languages of the parallel corpus, simultaneously. The quality of the tagging is rigorously evaluated for both languages of the corpora. The automatic unsupervised tagged data produced by SALAAM is further utilized to bootstrap a supervised learning WSD system, in essence, combining supervised and unsupervised approaches in an intelligent way to alleviate the resources acquisition bottleneck for supervised methods. Essentially, SALAAM is extended as an unsupervised approach for WSD within a learning framework; in many of the cases of the words disambiguated, SALAAM coupled with the machine learning system rivals the performance of a canonical supervised WSD system that relies on human tagged data for training. Realizing the fundamental role of similarity for SALAAM, we investigate different dimensions of semantic similarity as it applies to verbs since they are relatively more complex than nouns, which are the focus of the previous evaluations. We design a human judgment experiment to obtain human ratings on verbs' semantic similarity. The obtained human ratings are cast as a reference point for comparing different automated similarity measures that crucially rely on various sources of information. Finally, a cognitively salient model integrating human judgments in SALAAM is proposed as a means of improving its performance on sense disambiguation for verbs in particular and other word types in general.

[1]  Philip Resnik,et al.  Using Information Content to Evaluate Semantic Similarity in a Taxonomy , 1995, IJCAI.

[2]  Rada Mihalcea,et al.  Bootstrapping Large Sense Tagged Corpora , 2002, LREC.

[3]  Mary Laughren,et al.  Levels of Lexical Representation , 1993 .

[4]  Yorick Wilks,et al.  Cross-linguistic Discovery of Semantic Regularity , 2002 .

[5]  Bonnie J. Dorr,et al.  Large-Scale Dictionary Construction for Foreign Language Tutoring and Interlingual Machine Translation , 1998, Machine Translation.

[6]  H. Chertkow,et al.  Semantic memory , 2002, Current neurology and neuroscience reports.

[7]  Bonnie J. Dorr,et al.  Machine Translation: A View from the Lexicon , 1994, CL.

[8]  Philip Resnik,et al.  Word-level Alignment for Multilingual Resource Acquisition , 2002 .

[9]  B. Levin,et al.  From Lexical Semantics to Argument Realization , 1996 .

[10]  Genichiro Kikui,et al.  Resolving Translation Ambiguity using Non-parallel Bilingual Corpora , 1999 .

[11]  Julio Gonzalo,et al.  A Study of Polysemy and Sense Proximity in the Senseval-2 Test Suite , 2002, SENSEVAL.

[12]  Beth Levin,et al.  English Verb Classes and Alternations: A Preliminary Investigation , 1993 .

[13]  Dekang Lin,et al.  Dependency-Based Evaluation of Minipar , 2003 .

[14]  Hwee Tou Ng,et al.  Integrating Multiple Knowledge Sources to Disambiguate Word Sense: An Exemplar-Based Approach , 1996, ACL.

[15]  A. Nadas,et al.  A decision theorectic formulation of a training problem in speech recognition and a comparison of training by unconditional versus conditional maximum likelihood , 1983 .

[16]  ResnikPhilip,et al.  Distinguishing systems and distinguishing senses: new evaluation methods for Word Sense Disambiguation , 1999 .

[17]  Kareem Darwish,et al.  Building a Shallow Arabic Morphological Analyser in One Day , 2002, SEMITIC@ACL.

[18]  Hinrich Schütze,et al.  Automatic Word Sense Discrimination , 1998, Comput. Linguistics.

[19]  David Yarowsky,et al.  Unsupervised Word Sense Disambiguation Rivaling Supervised Methods , 1995, ACL.

[20]  Michael C. McCord,et al.  Slot Grammar: A System for Simpler Construction of Practical Natural Language Grammars , 1989, Natural Language and Logic.

[21]  Alon Itai,et al.  Word Sense Disambiguation Using a Second Language Monolingual Corpus , 1994, CL.

[22]  Dekang Lin,et al.  Using Syntactic Dependency as Local Context to Resolve Word Sense Ambiguity , 1997, ACL.

[23]  W. N. Locke,et al.  Machine Translation of Languages , 1956 .

[24]  Michael E. Lesk,et al.  Automatic sense disambiguation using machine readable dictionaries: how to tell a pine cone from an ice cream cone , 1986, SIGDOC '86.

[25]  P. Resnik Selection and information: a class-based approach to lexical relationships , 1993 .

[26]  Mona T. Diab,et al.  A statistical word-level translation model for comparable corpora , 2000 .

[27]  Philip Resnik,et al.  Selectional Preference and Sense Disambiguation , 1997 .

[28]  Philip Resnik,et al.  Disambiguating Noun Groupings with Respect to Wordnet Senses , 1995, VLC@ACL.

[29]  Bonnie J. Dorr,et al.  Spanish EuroWordNet and LCS-based interlingual MT , 1997, MTSUMMIT.

[30]  S. Pinker Learnability and Cognition: The Acquisition of Argument Structure , 1989 .

[31]  Adam Kilgarriff,et al.  Framework and Results for English SENSEVAL , 2000, Comput. Humanit..

[32]  I. Dan Melamed,et al.  Models of translation equivalence among words , 2000, CL.

[33]  Philip Resnik,et al.  Semantic Similarity in a Taxonomy: An Information-Based Measure and its Application to Problems of Ambiguity in Natural Language , 1999, J. Artif. Intell. Res..

[34]  Robert L. Mercer,et al.  A Statistical Approach to Sense Disambiguation in Machine Translation , 1991, HLT.

[35]  John D. Lafferty,et al.  Analysis, statistical transfer, and synthesis in machine translation , 1992, TMI.

[36]  Janyce Wiebe,et al.  Word-Sense Disambiguation Using Decomposable Models , 1994, ACL.

[37]  Dekang Lin,et al.  Automatic Retrieval and Clustering of Similar Words , 1998, ACL.

[38]  Hermann Ney,et al.  Improved Statistical Alignment Models , 2000, ACL.

[39]  Philip Resnik,et al.  Evaluating Translational Correspondence using Annotation Projection , 2002, ACL.

[40]  Philip Resnik,et al.  Mining the Web for Bilingual Text , 1999, ACL.

[41]  Kenneth Ward Church,et al.  Using bilingual materials to develop word sense disambiguation methods , 1992, TMI.

[42]  EstimationPeter,et al.  The Mathematics of Machine Translation : Parameter , 2004 .

[43]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[44]  Ted Pedersen,et al.  Extended Gloss Overlaps as a Measure of Semantic Relatedness , 2003, IJCAI.

[45]  Ted Pedersen Machine Learning with Lexical Features: The Duluth Approach to SENSEVAL-2 , 2001, SENSEVAL@ACL.

[46]  Philip Resnik,et al.  The Bible as a Parallel Corpus: Annotating the ‘Book of 2000 Tongues’ , 1999, Comput. Humanit..

[47]  Edward F. Kelly,et al.  Computer recognition of English word senses , 1975 .

[48]  I. D. Melamed Measuring Semantic Entropy , 1997 .

[49]  Miriam Butt,et al.  The Projection of Arguments: Lexical and Compositional Factors , 2004 .

[50]  Philip Resnik,et al.  Supervised Sense Tagging using Support Vector Machines , 2001, *SEMEVAL.

[51]  John Cocke,et al.  A Statistical Approach to Machine Translation , 1990, CL.

[52]  Eneko Agirre,et al.  Combining Supervised and Unsupervised Lexical Knowledge Methods for Word Sense Disambiguation , 2000, Comput. Humanit..

[53]  Mona T. Diab,et al.  An Unsupervised Method for Multilingual Word Sense Tagging Using Parallel Corpora , 2000, ACL 2000.

[54]  Steven Lawrence Small,et al.  Word expert parsing: a theory of distributed word-based natural language understanding , 1980 .

[55]  Rada Mihalcea,et al.  A Method for Word Sense Disambiguation of Unrestricted Text , 1999, ACL.

[56]  Julio Gonzalo,et al.  Towards a Universal Index of Meaning , 1999 .

[57]  James H. Martin,et al.  Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition, 2nd Edition , 2000, Prentice Hall series in artificial intelligence.

[58]  Philip J. Hayes On Semantic Nets, Frames and Associations , 1977, IJCAI.

[59]  David Yarowsky,et al.  Word-Sense Disambiguation Using Statistical Models of Roget’s Categories Trained on Large Corpora , 2010, COLING.

[60]  Rada Mihalcea,et al.  Word Sense Disambiguation based on Semantic Density , 1998, WordNet@ACL/COLING.

[61]  David Yarowsky,et al.  Statistical Machine Translation: Final Report , 1999 .

[62]  Thorsten Joachims,et al.  Text Categorization with Support Vector Machines: Learning with Many Relevant Features , 1998, ECML.

[63]  Adam Kilgarriff,et al.  SENSEVAL: an exercise in evaluating world sense disambiguation programs , 1998, LREC.

[64]  Nancy Ide,et al.  Introduction to the Special Issue on Word Sense Disambiguation: The State of the Art , 1998, Comput. Linguistics.