论文信息 - POC-NLW Template Based Tagging Method for Chinese Word Segmentation

POC-NLW Template Based Tagging Method for Chinese Word Segmentation

In Chinese word segmentation, disambiguation and unknown words identification are becoming the two key issues. In this paper, a two-stage strategy based system is constructed to deal with these problems. First, an n-gram based model is applied to do the basic segmentation as well as disambiguation in some extent. Then, in the second stage, a language tagging template, named POC-NLW, is adopted to carry out a character sequence tagging procedure based on hidden Markov model, which is used to refine the results from the first stage and to identify unknown words. Several detailed experiments have been implemented on the SIGHAN Bakeoff 2005 corpus. Experimental results show that the method can achieve high accuracy on word segmentation, as well as on unknown words identification, with appreciable processing efficiency. This method is characterized by the good interoperability and expansionary over different kinds of unknown words, thus it is applicable for practical Chinese information processing applications

[1] Tetsuji Nakagawa,et al. Chinese and Japanese Word Segmentation Using Word-Level and Character-Level Information , 2004, COLING.

[2] Andi Wu,et al. Statistically-Enhanced New Word Identification in a Rule-Based Chinese System , 2000, ACL 2000.

[3] Thorsten Brants,et al. TnT – A Statistical Part-of-Speech Tagger , 2000, ANLP.

[4] Eugene Charniak,et al. Equations for Part-of-Speech Tagging , 1993, AAAI.

[5] Guohong Fu,et al. A Two-stage Statistical Word Segmentation System for Chinese , 2003, SIGHAN.

[6] Hwee Tou Ng,et al. A Maximum Entropy Approach to Chinese Word Segmentation , 2005, SIGHAN@IJCNLP 2005.

[7] Nianwen Xu,et al. Chinese Word Segmentation as Character Tagging , 2003, Int. J. Comput. Linguistics Chin. Lang. Process..

[8] Yuji Matsumoto,et al. Combining Segmenter and Chunker for Chinese Word Segmentation , 2003, SIGHAN.

[9] Nianwen Xue,et al. Chinese Word Segmentation as LMR Tagging , 2003, SIGHAN.

[10] Frederick Jelinek,et al. Interpolated estimation of Markov source parameters from sparse data , 1980 .