论文信息 - Using Unsupervised Paradigm Acquisition for Prefixes

Using Unsupervised Paradigm Acquisition for Prefixes

We describe a simple method of unsupervised morpheme segmentation of words in an unknown language. All that is needed is a raw text corpus (or a list of words) in the given language. The algorithm identifies word parts occurring in many words and interprets them as morpheme candidates (prefixes, stems and suffixes). New treatment of prefixes is the main innovation in comparison to [1]. After filtering out spurious hypotheses, the list of morphemes is applied to segment input words. Official Morpho Challenge 2008 evaluation is given together with some additional experiments. Processing of prefixes improved the F-score by 5 to 11 points for German, Finnish and Turkish, while it failed to improve English and Arabic. We also analyze and discuss errors with respect to the evaluation method.

Daniel Zeman

[1] Erwin Chan. Towards Unsupervised Induction of Morphophonemic Rules , 2007, CLEF.

[2] Stefan Bordag. Unsupervised and Knowledge-free Morpheme Segmentation and Analysis , 2007, CLEF.

[3] Carol Peters,et al. Advances in Multilingual and Multimodal Information Retrieval, 8th Workshop of the Cross-Language Evaluation Forum, CLEF 2007, Budapest, Hungary, September 19-21, 2007, Revised Selected Papers , 2008, CLEF.

[4] Alon Lavie,et al. ParaMor: Finding Paradigms across Morphology , 2008, CLEF.

[5] James Mayfield,et al. N-Gram Morphemes for Retrieval , 2007, CLEF.

[6] Michael Tepper. Using Hand-Written Rewrite Rules to Induce Underlying Morphology , 2007, CLEF.

[7] Mikko Kurimo,et al. Overview of Morpho Challenge 2008 , 2008, CLEF.

[8] T. Wassmer. 6 , 1900, EXILE.

[9] Daniel Zeman. Unsupervised Acquiring of Morphological Paradigms from Tokenized Text , 2007, CLEF.

[10] Delphine Bernhard,et al. Simple Morpheme Labelling in Unsupervised Morpheme Analysis , 2008, CLEF.

[11] Ville T. Turunen,et al. Advances in Multilingual and MultiModal Information Retrieval, 9th Workshop of the Cross-Language Evaluation Forum, CLEF 2008, Aarhus, Denmark, September 17-19, 2008, Revised Selected Papers , 2009 .