论文信息 - Unsupervised Lexicon-Based Resolution of Unknown Words for Full Morphological Analysis

Unsupervised Lexicon-Based Resolution of Unknown Words for Full Morphological Analysis

Morphological disambiguation proceeds in 2 stages: (1) an analyzer provides all possible analyses for a given token and (2) a stochastic disambiguation module picks the most likely analysis in context. When the analyzer does not recognize a given token, we hit the problem of unknowns. In large scale corpora, unknowns appear at a rate of 5 to 10% (depending on the genre and the maturity of the lexi

[1] Andrei Mikheev,et al. Automatic Rule Induction for Unknown-Word Guessing , 1997, CL.

[2] Daniel Jurafsky,et al. Automatic Tagging of Arabic Text: From Raw Text to Base Phrase Chunks , 2004, NAACL.

[3] Nizar Habash,et al. MAGEAD: A Morphological Analyzer and Generator for the Arabic Dialects , 2006, ACL.

[4] Richard M. Schwartz,et al. Coping with Ambiguity and Unknown Words through Probabilistic Models , 1993, CL.

[5] Khalil Sima'an,et al. Choosing an Optimal Architecture for Segmentation and POS-Tagging of Modern Hebrew , 2005, SEMITIC@ACL.

[6] Khalil Sima'an,et al. Smoothing a Lexicon-based POS Tagger for Arabic and Hebrew , 2007, SEMITIC@ACL.

[7] George K. Kokkinakis,et al. Automatic Stochastic Tagging of Natural Language Texts , 1995, Comput. Linguistics.

[8] Alon Itai,et al. Learning Morpho-Lexical Probabilities from an Untagged Corpus with an Application to Hebrew , 1995, CL.

[9] Tetsuji Nakagawa,et al. Chinese and Japanese Word Segmentation Using Word-Level and Character-Level Information , 2004, COLING.