Information theoretic spectral clustering

We discuss a new information-theoretic framework for spectral clustering that is founded on the recently introduced information cut. A novel spectral clustering algorithm is proposed, where the clustering solution is given as a linearly weighted combination of certain top eigenvectors of the data affinity matrix. The information cut provides us with a theoretically well-defined graph-spectral cost function, and also establishes a close link between spectral clustering, and non-parametric density estimation. As a result, a natural criterion for creating the data affinity matrix is provided. We present preliminary clustering results to illustrate some of the properties of our algorithm, and we also make comparative remarks.

[1]  Sudeep Sarkar,et al.  Supervised Learning of Large Perceptual Organization: Graph Spectral Partitioning and Learning Automata , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[3]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[4]  Santosh S. Vempala,et al.  On clusterings-good, bad and spectral , 2000, Proceedings 41st Annual Symposium on Foundations of Computer Science.

[5]  C. D. Kemp,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[6]  Pietro Perona,et al.  A Factorization Approach to Grouping , 1998, ECCV.

[7]  Andrew B. Kahng,et al.  Fast spectral methods for ratio cut partitioning and clustering , 1991, 1991 IEEE International Conference on Computer-Aided Design Digest of Technical Papers.

[8]  Andrew B. Kahng,et al.  New spectral methods for ratio cut partitioning and clustering , 1991, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[9]  Robert Jenssen,et al.  Information cut and information forces for clustering , 2003, 2003 IEEE XIII Workshop on Neural Networks for Signal Processing (IEEE Cat. No.03TH8718).

[10]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[11]  Yair Weiss,et al.  Segmentation using eigenvectors: a unifying view , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[12]  Nello Cristianini,et al.  Spectral Kernel Methods for Clustering , 2001, NIPS.

[13]  Gene H. Golub,et al.  Matrix computations , 1983 .

[14]  Chris H. Q. Ding,et al.  A min-max cut algorithm for graph partitioning and data clustering , 2001, Proceedings 2001 IEEE International Conference on Data Mining.

[15]  Lihao Xu,et al.  Multiway cuts and spec-tral clustering , 2003 .

[16]  B. Silverman.  Density estimation for statistics and data analysis , 1986 .