Text classification with self-organizing maps: Some lessons learned

Abstract The self-organizing map has already found appreciation for document classification in the information retrieval community. The map display is a highly effective and intuitive metaphor for orientation in the information space established by a document collection. In this paper we discuss ways for using self-organizing maps for document classification. Furthermore, we argue in favor of paying more attention to the fact that document collections lend themselves naturally to a hierarchical structure defined by the subject matter of the documents. We take advantage of this fact by using a hierarchically organized neural network, built up from a number of independent self-organizing maps in order to enable the true establishment of a document taxonomy. As a highly convenient side effect of using such an architecture, the time needed for training is reduced substantially and the user is provided with an even more intuitive metaphor for visualization. Since the single layers of self-organizing maps represent different aspects of the document collection at different levels of detail, the neural network shows the document collection in a form comparable to an atlas where the user may easily select the most appropriate degree of granularity depending on the actual focus of interest during the exploration of the document collection.