Fuzzy clustering with Multiple Kernels

In this paper, the kernel fuzzy c-means clustering algorithm is extended to an adaptive cluster model which maps data points to a high dimensional feature space through an optimal convex combination of homogenous kernels with respect to each cluster. This generalized model, called Fuzzy C-Means with Multiple Kernels (FCM-MK), strives to find a good partitioning of the data into meaningful clusters and the optimal kernel-induced feature map in a completely unsupervised way. It constructs the kernel from a number of Gaussian kernels and learns a resolution specific weight for each kernel function in each cluster. This allows better characterization and adaptability to each individual cluster. The effectiveness of the proposed algorithm is demonstrated for several toy and real data sets.

[1]  Gunnar Rätsch,et al.  An introduction to kernel-based learning algorithms , 2001, IEEE Trans. Neural Networks.

[2]  Olivier Bousquet,et al.  On the Complexity of Learning the Kernel Matrix , 2002, NIPS.

[3]  Jianqiang Yi,et al.  Design of interval type-2 fuzzy logic system using sampled data and priorknowledge , 2009 .

[4]  Tae-Wook Kim,et al.  Parameter conditions and least squares identification of single-input single-output convex fuzzy system , 2007, SICE Annual Conference 2007.

[5]  Michael I. Jordan,et al.  Multiple kernel learning, conic duality, and the SMO algorithm , 2004, ICML.

[6]  Jack L. Meador,et al.  Encoding a priori information in feedforward networks , 1991, Neural Networks.

[7]  Jerry M. Mendel,et al.  A quantitative comparison of interval type-2 and type-1 fuzzy logic systems: First results , 2010, International Conference on Fuzzy Systems.

[8]  Kazuo Tanaka,et al.  Modeling and control of carbon monoxide concentration using a neuro-fuzzy technique , 1995, IEEE Trans. Fuzzy Syst..

[9]  Wenjian Wang,et al.  Determination of the spread parameter in the Gaussian kernel for classification and regression , 2003, Neurocomputing.

[10]  Miin-Shen Yang,et al.  Alternative c-means clustering algorithms , 2002, Pattern Recognit..

[11]  Jianqiang Yi,et al.  Analysis and design of monotonic type-2 fuzzy inference systems , 2009, 2009 IEEE International Conference on Fuzzy Systems.

[12]  Sayan Mukherjee,et al.  Choosing Multiple Parameters for Support Vector Machines , 2002, Machine Learning.

[13]  Jerry M. Mendel,et al.  Equalization of nonlinear time-varying channels using type-2 fuzzy adaptive filters , 2000, IEEE Trans. Fuzzy Syst..

[14]  Jinwook Kim,et al.  Single-input single-output convex fuzzy systems as universal approximators for single-input single-output convex functions , 2009, 2009 IEEE International Conference on Fuzzy Systems.

[15]  Jieping Ye,et al.  Learning the kernel matrix in discriminant analysis via quadratically constrained quadratic programming , 2007, KDD '07.

[16]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[17]  Mark A. Girolami,et al.  Mercer kernel-based clustering in feature space , 2002, IEEE Trans. Neural Networks.

[18]  Vladimir Vapnik,et al.  The Nature of Statistical Learning , 1995 .

[19]  Jerry M. Mendel,et al.  Advances in type-2 fuzzy sets and systems , 2007, Inf. Sci..

[20]  Witold Pedrycz,et al.  Kernel-based fuzzy clustering and fuzzy clustering: A comparative experimental study , 2010, Fuzzy Sets Syst..

[21]  Alexander J. Smola,et al.  Learning the Kernel with Hyperkernels , 2005, J. Mach. Learn. Res..

[22]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2004 .

[23]  Hichem Frigui,et al.  Clustering and aggregation of relational data with applications to image database categorization , 2007, Pattern Recognit..

[24]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[25]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[26]  Jianqiang Yi,et al.  The monotonicity and convexity of unnormalized interval type-2 TSK Fuzzy Logic Systems , 2010, International Conference on Fuzzy Systems.

[27]  Lennart Ljung,et al.  Ensuring monotonic gain characteristics in estimated models by fuzzy model structures , 2000, Autom..

[28]  Jian-Huang Lai,et al.  Kernel subspace LDA with optimized kernel parameters on face recognition , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[29]  Zhongdong Wu,et al.  Fuzzy C-means clustering algorithm based on kernel method , 2003, Proceedings Fifth International Conference on Computational Intelligence and Multimedia Applications. ICCIMA 2003.

[30]  Jin S. Lee,et al.  Parameter conditions for monotonic Takagi-Sugeno-Kang fuzzy system , 2002, Fuzzy Sets Syst..

[31]  Nello Cristianini,et al.  Learning the Kernel Matrix with Semidefinite Programming , 2002, J. Mach. Learn. Res..

[32]  N. Cristianini,et al.  On Kernel-Target Alignment , 2001, NIPS.

[33]  Yves Grandvalet,et al.  More efficiency in multiple kernel learning , 2007, ICML '07.