Research on Chinese Word Segmentation Algorithm Based on Special Identifiers
暂无分享,去创建一个
Chinese information processing is a tedious and massive information processing engineering, Chinese word processing is that the whole project-based and one among the important aspects. This paper provides a word segmentation method based on special identifiers, and realizes a word segmentation system by combining the special identifier set with the modified two-character dictionary structure, before it carries out the comparison test for that system and other word segmentation systems by SOUGOU training corpus’s test text.
[1] Cui Hong-yan. Research on an improved Chinese segmentation algorithm based on word frequency statistic , 2008 .
[2] Zhang Ke. Multi-hash indexing algorism for Chinese character segmentation , 2007 .
[3] Hao Tian-yong. The State of the Art and Difficulties in Automatic Chinese Word Segmentation , 2005 .
[4] Wu Jing. Fast dictionary mechanism for Chinese word segmentation , 2009 .