Applying the Possibilistic c-Means Algorithm in Kernel-Induced Spaces

In this paper, we study a kernel extension of the classic possibilistic c-means. In the proposed extension, we implicitly map input patterns into a possibly high-dimensional space by means of positive semidefinite kernels. In this new space, we model the mapped data by means of the possibilistic clustering algorithm. We study in more detail the special case where we model the mapped data using a single cluster only, since it turns out to have many interesting properties. The modeled memberships in kernel-induced spaces yield a modeling of generic shapes in the input space. We analyze in detail the connections to one-class support vector machines and kernel density estimation, thus, suggesting that the proposed algorithm can be used in many scenarios of unsupervised learning. In the experimental part, we analyze the stability and the accuracy of the proposed algorithm on some synthetic and real datasets. The results show high stability and good performances in terms of accuracy.

[1]  Kevin Baker,et al.  Classification of radar returns from the ionosphere using neural networks , 1989 .

[2]  B. Silverman Density estimation for statistics and data analysis , 1986 .

[3]  A. Atiya,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[4]  M. Aizerman,et al.  Theoretical Foundations of the Potential Function Method in Pattern Recognition Learning , 1964 .

[5]  Maurizio Filippone,et al.  Dealing with non-metric dissimilarities in fuzzy central clustering algorithms , 2009, Int. J. Approx. Reason..

[6]  R. Fisher THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[7]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[8]  Mauro Barni,et al.  Comments on "A possibilistic approach to clustering" , 1996, IEEE Trans. Fuzzy Syst..

[9]  고한석,et al.  컬러 영상에서 Support Vector Domain Description을 이용한 얼굴 검출 , 2005 .

[10]  P. Hall,et al.  Kernel density estimation with spherical data , 1987 .

[11]  James M. Keller,et al.  The possibilistic C-means algorithm: insights and recommendations , 1996, IEEE Trans. Fuzzy Syst..

[12]  Daoqiang Zhang,et al.  Fuzzy clustering using kernel method , 2002 .

[13]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[14]  Hava T. Siegelmann,et al.  Support Vector Clustering , 2002, J. Mach. Learn. Res..

[15]  Francesco Masulli,et al.  A survey of kernel and spectral methods for clustering , 2008, Pattern Recognit..

[16]  Robert P. W. Duin,et al.  Support vector domain description , 1999, Pattern Recognit. Lett..

[17]  Song-can Chen,et al.  Kernel-based fuzzy and possibilistic c-means clustering , 2003 .

[18]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[19]  Joachim M. Buhmann,et al.  Stability-Based Validation of Clustering Solutions , 2004, Neural Computation.

[20]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[21]  Gunnar Rätsch,et al.  Input space versus feature space in kernel-based methods , 1999, IEEE Trans. Neural Networks.

[22]  N. Aronszajn Theory of Reproducing Kernels. , 1950 .

[23]  Frank Klawonn,et al.  A contribution to convergence theory of fuzzy c-means and derivatives , 2003, IEEE Trans. Fuzzy Syst..

[24]  James M. Keller,et al.  A possibilistic approach to clustering , 1993, IEEE Trans. Fuzzy Syst..

[25]  Francesco Masulli,et al.  Possibilistic Clustering in Feature Space , 2007, WILF.

[26]  O. Mangasarian,et al.  Multisurface method of pattern separation for medical diagnosis applied to breast cytology. , 1990, Proceedings of the National Academy of Sciences of the United States of America.