Cooperative and penalized competitive learning with application to kernel-based clustering

Abstract Competitive learning approaches with individual penalization or cooperation mechanisms have the attractive ability of automatic cluster number selection in unsupervised data clustering. In this paper, we further study these two mechanisms and propose a novel learning algorithm called Cooperative and Penalized Competitive Learning (CPCL), which implements the cooperation and penalization mechanisms simultaneously in a single competitive learning process. The integration of these two different kinds of competition mechanisms enables the CPCL to locate the cluster centers more quickly and be insensitive to the number of seed points and their initial positions. Additionally, to handle nonlinearly separable clusters, we further introduce the proposed competition mechanism into kernel clustering framework. Correspondingly, a new kernel-based competitive learning algorithm which can conduct nonlinear partition without knowing the true cluster number is presented. The promising experimental results on real data sets demonstrate the superiority of the proposed methods.

[1]  Hong Yan,et al.  Pattern recognition techniques for the emerging field of bioinformatics: A review , 2005, Pattern Recognit..

[2]  Francesco Masulli,et al.  A survey of kernel and spectral methods for clustering , 2008, Pattern Recognit..

[3]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[4]  Changshui Zhang,et al.  Competitive EM algorithm for finite mixture models , 2004, Pattern Recognit..

[5]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[6]  Gunnar Rätsch,et al.  An introduction to kernel-based learning algorithms , 2001, IEEE Trans. Neural Networks.

[7]  J. MacQueen Some methods for classification and analysis of multivariate observations , 1967 .

[8]  Yiu-ming Cheung,et al.  Maximum weighted likelihood via rival penalized EM for density mixture clustering with automatic model selection , 2005, IEEE Transactions on Knowledge and Data Engineering.

[9]  Hichem Frigui,et al.  A Robust Competitive Clustering Algorithm With Applications in Computer Vision , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Erkki Oja,et al.  Rival penalized competitive learning for clustering analysis, RBF net, and curve detection , 1993, IEEE Trans. Neural Networks.

[11]  Aristidis Likas,et al.  The Global Kernel $k$-Means Algorithm for Clustering in Feature Space , 2009, IEEE Transactions on Neural Networks.

[12]  Jinwen Ma,et al.  A cost-function approach to rival penalized competitive learning (RPCL) , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[13]  Davide Bacciu,et al.  Competitive Repetition Suppression (CoRe) Clustering: A Biologically Inspired Learning Model With Application to Robust Clustering , 2008, IEEE Transactions on Neural Networks.

[14]  Sadaaki Miyamoto,et al.  LVQ clustering and SOM using a kernel function , 2004, 2004 IEEE International Conference on Fuzzy Systems (IEEE Cat. No.04CH37542).

[15]  Yiu-ming Cheung A competitive and cooperative learning approach to robust data clustering , 2004, Neural Networks and Computational Intelligence.

[16]  Greg Hamerly,et al.  Learning the k in k-means , 2003, NIPS.

[17]  Colin Fyfe,et al.  The kernel self-organising map , 2000, KES'2000. Fourth International Conference on Knowledge-Based Intelligent Engineering Systems and Allied Technologies. Proceedings (Cat. No.00TH8516).

[18]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[19]  LiewAlan Wee-Chung,et al.  Pattern recognition techniques for the emerging field of bioinformatics , 2005 .

[20]  Stanley C. Ahalt,et al.  Competitive learning algorithms for vector quantization , 1990, Neural Networks.

[21]  Tao Li,et al.  Cooperation Controlled Competitive Learning Approach for Data Clustering , 2008, 2008 International Conference on Computational Intelligence and Security.