论文信息 - Research on Hybrid Index for Chinese IR

Research on Hybrid Index for Chinese IR

It is essential to identify terms that are used as index units in the processing of Chinese documents and queries in IR. In this paper new kinds of hybrid index are proposed, which combine words and bigrams. This kind of hybrid index can reduce the impact of out-of-vocabulary and segmentation ambiguity for Chinese IR, because the dictionary is applied to detect segmentation ambiguities in a flexible way rather than by the ambiguity table rigidly. The experiments show the new kind of hybrid index is not only comparable with bigrams indexing, but also enhances the retrieval efficiency.

Tiejun Zhao | Chen Chen | Muyun Yang | Haoliang Qi | Sheng Li

[1] Kam-Fai Wong,et al. Hybrid Term Indexing: an Evaluation , 2001, NTCIR.

[2] Kam-Fai Wong,et al. Hybrid Term Indexing for Weighted Boolean and Vector Space Models , 2001, Int. J. Comput. Process. Orient. Lang..

[3] K Tsou,et al. The Role of High Frequent Maximal Crossing Ambiguities in Chinese Word Segmentation , 1999 .

[4] Hong Zhou,et al. Preliminary Qualitative Analysis of Segmented vs Bigram Indexing in Chinese , 1997, TREC.

[5] Jian Zhang,et al. On the use of words and n-grams for Chinese information retrieval , 2000, IRAL '00.

[6] Kui-Lam Kwok. Comparing representations in Chinese information retrieval , 1997, SIGIR '97.

[7] Kam-Fai Wong,et al. Hybrid term indexing for different IR models , 2000, IRAL '00.

[8] Kui-Lam Kwok,et al. A comparison of Chinese document indexing strategies and retrieval models , 2002, TALIP.

[9] Ying Xiong,et al. A New Machine Learning Method for Chinese Overlapping Disambiguity--Conditional Random Fields , 2007, 2007 International Conference on Machine Learning and Cybernetics.

[10] Zhao Tie. Increasing Accuracy of Chinese Segmentation with Strategy of Multi step Processing , 2001 .

[11] Ma Ting-ting. Overview of Chinese Automatic Word Segmentation System , 2010 .

[12] Jian-Yun Nie,et al. Chinese information retrieval: using characters or words? , 1999, Inf. Process. Manag..