论文信息 - Enhancement of DTP Feature Selection Method for Text Categorization

Enhancement of DTP Feature Selection Method for Text Categorization

This paper studies the structure of vectors obtained by using term selection methods in high-dimensional text collection. We found that the distance to transition point (DTP) method omits commonly occurring terms, which are poor discriminators between documents, but which convey important information about a collection. Experimental results obtained on the Reuters-21578 collection with the k-NN classifier show that feature selection by DTP combined with common terms outperforms slightly simple document frequency.

Héctor Jiménez-Salazar | Edgar Moyotl-Hernández

[1] Héctor Jiménez-Salazar,et al. An analysis on Frequency of terms for Text Categorization , 2004, Proces. del Leng. Natural.

[2] Gerard Salton,et al. A vector space model for automatic indexing , 1975, CACM.

[3] Fabrizio Sebastiani,et al. Machine learning in automated text categorization , 2001, CSUR.

[4] Yiming Yang,et al. A re-examination of text categorization methods , 1999, SIGIR '99.

[5] Maria Simi,et al. Experiments on the Use of Feature Selection and Negative Evidence in Automated Text Categorization , 2000, ECDL.

[6] Andrew Donald Booth,et al. A "Law" of Occurrences for Words of Low Frequency , 1967, Inf. Control..

[7] Yiming Yang,et al. A Comparative Study on Feature Selection in Text Categorization , 1997, ICML.

[8] Héctor Jiménez Salazar,et al. Experiments in Text Categorization Using Term Selection by Distance to Transition Point , 2004 .

[9] Fabrizio Sebastiani,et al. An Analysis of the Relative Difficulty of Reuters-21578 Subsets , 2004, LREC.

[10] George Kingsley Zipf,et al. Human behavior and the principle of least effort , 1949 .

[11] Héctor Jiménez Salazar,et al. An analysis on frequency of terms for text categorization , 2004 .

[12] Inderjit S. Dhillon,et al. Concept Decompositions for Large Sparse Text Data Using Clustering , 2004, Machine Learning.