论文信息 - Stemming and Segmentation for Classical Tibetan

Stemming and Segmentation for Classical Tibetan

Tibetan is a monosyllabic language for which computerized language tools are largely lacking. We describe the development of a syllable stemmer for Tibetan. The stemmer is based on a set of rules that strive to identify the vowel, the core letter of the syllable, and then the other parts. We demonstrate the value of the stemmer with two applications: determining stem similarity of two syllables and word segmentation. Our stemmer is being made available as an open-source tool and word segmentation as a freely-available online tool.

[1] George Kingsley Zipf,et al. Human Behaviour and the Principle of Least Effort: an Introduction to Human Ecology , 2012 .

[2] T. Wylie,et al. A Standard System of Tibetan Transcription , 1959 .

[3] M. Hahn. Lehrbuch der klassischen tibetischen Schriftsprache , 1985 .

[4] Jürgen Schmidhuber,et al. Long Short-Term Memory , 1997, Neural Computation.

[5] Corinna Cortes,et al. Support-Vector Networks , 1995, Machine Learning.

[6] Heming Huang,et al. General Structure Based Collation of Tibetan Syllables , 2010 .

[7] Jian Wu,et al. Zipf’s Law and Statistical Data on Modern Tibetan , 2014, COLING.

[8] Lior Wolf,et al. Finding Inexact Quotations Within a Tibetan Buddhist Corpus , 2014, DH.

[9] Jimmy Ba,et al. Adam: A Method for Stochastic Optimization , 2014, ICLR.

[10] Xuanjing Huang,et al. Long Short-Term Memory Neural Networks for Chinese Word Segmentation , 2015, EMNLP.