Dimension Reduction for Text Data Representation Based on Cluster Structure Preserving Projection

In today’s vector space information retrieval systems, dim ension reduction is imperative for efficiently manipulating the massive quantity of data. To be useful, this l ower dimensional representation must be a good approximation of the full document set. To that end, we adapt and extend the discriminant analysis projection used in pattern recognition. This projection prese ves cluster structure by maximizing the scatter between clusters while minimizing the scatter within clust er . A limitation of discriminant analysis is that one of its scatter matrices must be nonsingular, which restr icts its application to document sets in which the number of terms does not exceed the number of documents. We sh ow that by using the generalized singular value decomposition (GSVD), we can achieve the same goal reg rdless of the relative dimensions of our data. We also show that, for k clusters, the generalized right singular vectors that corr espond to thek 1 largest generalized singular values are all we need to compu te the optimal transformation to the reduced dimension. In addition, applying the GSVD allows us to avoid the explicit formation of the scatter matrices in favor of working directly with the data matrix, thus impro ving the numerical properties of the approach. The work of all three authors was supported in part by the Nati on l Science Foundation grant CCR-9901992. Dept. of Computer Science and Engineering, Univ. of Minnesota, Minn eapolis, MN 55455, U.S.A.( hpark@cs.umn.edu) yDept. of Computer Science and Engineering, Univ. of Minneso ta, Minneapolis, MN 55455, U.S.A.( jeon@cs.umn.edu) zDepartment of Computer Science and Engineering, Univ. of Mi nnesota, Minneapolis, MN 55455, U.S.A.(howland@cs.umn.edu).