Indefinite Parzen Window for Spectral Clustering

Kernel functions used to compute pairwise similarities in spectral clustering and kernel methods are almost exclusively positive semidefinite. We show that for a recent information theoretic spectral clustering algorithm, the theoretically optimal kernel is given by the indefinite Epanechnikov function via Parzen windowing for density estimation. We perform clustering using this indefinite kernel, and report results which show improved performance. Finally, we briefly indicate some properties of the negative spectrum of the corresponding indefinite kernel matrix.

[1]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[2]  Robert Jenssen,et al.  Information cut and information forces for clustering , 2003, 2003 IEEE XIII Workshop on Neural Networks for Signal Processing (IEEE Cat. No.03TH8718).

[3]  Ali H. Sayed,et al.  Linear Estimation in Krein Spaces - Part I: Theory , 1996 .

[4]  Robert Jenssen,et al.  Information Theoretic Angle-Based Spectral Clustering: A Theoretical Analysis and an Algorithm , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[5]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis: List of kernels , 2004 .

[6]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[7]  T. Kailath,et al.  Linear estimation in Krein spaces. I. Theory , 1996, IEEE Trans. Autom. Control..

[8]  J. Mercer Functions of positive and negative type, and their connection with the theory of integral equations , 1909 .

[9]  Alexander J. Smola,et al.  Learning with non-positive kernels , 2004, ICML.

[10]  Jitendra Malik,et al.  Spectral Partitioning with Indefinite Kernels Using the Nyström Extension , 2002, ECCV.

[11]  E. Parzen On Estimation of a Probability Density Function and Mode , 1962 .

[12]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  Joachim M. Buhmann,et al.  On the information and representation of non-Euclidean pairwise data , 2006, Pattern Recognit..

[14]  Deniz Erdogmus,et al.  Information Theoretic Learning , 2005, Encyclopedia of Artificial Intelligence.