Limbo: A scalable algorithm to cluster categorical data

Clustering is a problem of great practical importance in numerous applications. The problem of clustering becomes more challenging when the data is categorical, that is, when there is no inherent distance measure between data values. In this work, we introduce LIMBO, a scalable hierarchical categorical clustering algorithm that builds on the Information Bottleneck (IB) framework for quantifying the relevant information preserved when clustering. We use the IB framework to define a distance measure for categorical tuples and we also present a novel distance measure for categorical attribute values. We show how the LIMBO algorithm can be used to cluster both tuples and attribute values. LIMBO handles large data sets by producing a summary model for the data. We propose two different versions of LIMBO, where we either control the size or the accuracy of the model. We present an experimental evaluation of both versions of LIMBO, and we study how clustering quality in information theoretic clustering algorithms compares to other categorical clustering algorithms. LIMBO also supports a tradeoff between efficiency (in terms of space and time). We quantify this trade-off and we demonstrate that LIMBO allows for substantial improvements in efficiency with negligible decrease in quality. LIMBO is a hierarchical algorithm that produces clusterings for a range of k values (where k is the number of clusters). We take advantage of this feature to examine heuristics for selecting good clusterings (with natural values of k) within this range. 1

[1]  Jon M. Kleinberg,et al.  Clustering categorical data: an approach based on dynamical systems , 2000, The VLDB Journal.

[2]  Mark A. Gluck,et al.  Information, Uncertainty and the Utility of Categories , 1985 .

[3]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[4]  Yi Li,et al.  COOLCAT: an entropy-based algorithm for categorical clustering , 2002, CIKM '02.

[5]  Naftali Tishby,et al.  The information bottleneck method , 2000, ArXiv.

[6]  Sudipto Guha,et al.  ROCK: a robust clustering algorithm for categorical attributes , 1999, Proceedings 15th International Conference on Data Engineering (Cat. No.99CB36337).

[7]  Tian Zhang,et al.  BIRCH: an efficient data clustering method for very large databases , 1996, SIGMOD '96.

[8]  Yao Wang,et al.  A robust and scalable clustering algorithm for mixed type attributes in large database environment , 2001, KDD '01.

[9]  Naftali Tishby,et al.  Agglomerative Information Bottleneck , 1999, NIPS.

[10]  Heikki Mannila,et al.  Context-Based Similarity Measures for Categorical Databases , 2000, PKDD.

[11]  Naftali Tishby,et al.  Unsupervised document classification using sequential information maximization , 2002, SIGIR '02.

[12]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[13]  Allan Borodin,et al.  Finding authorities and hubs from link structures on the World Wide Web , 2001, WWW '01.

[14]  Naftali Tishby,et al.  Document clustering using word clusters via the information bottleneck method , 2000, SIGIR '00.

[15]  Johannes Gehrke,et al.  CACTUS—clustering categorical data using summaries , 1999, KDD '99.