Clustering by means of unsupervised decision trees or hierarchical and K-means-like algorithm

A classical information retrieval system returns a list of documents to a user query. The answer list is often so long that users cannot explore all the documents retrieved. A classification of the retrieved documents allows to thematically organize them and to improve precision. In this paper, we present and compare two text classification algorithms. The first one is a clustering algorithm (K-Means-like) initialized with a partial hierarchical classification. The second one is a new algorithm that relies on unsupervised decision trees (UDTs). The indexing methods we use (TF-IDF weighting scheme, cosine similarity in the vector space model) prevent from really considering all the subjects dealt with in the texts. A better way to take all the themes into account is to cluster sentences from documents instead of documents as a whole. This is achieved the second method we propose. The effectiveness of these methods is evaluated over Amaryllis'99 corpora and queries. Since these methods are applied during a post-processing phase, they can be used with any IR-system which returns a list of documents. The methods presented here allow to obtain significant results and improvement compared with a search without classification In order to verify that improvement is due to the methods and not to the sharing out of items into classes, the results obtained are compared with those of a random classification.