Sparse learning based fuzzy c-means clustering

Recently sparse representation (SR) based clustering has attracted a growing interests in the field of image processing and pattern recognition. Since the SR technology has favorable category distinguishing ability, we introduce it into the fuzzy clustering in this paper, and propose a new clustering algorithm, called sparse learning based fuzzy c-means (SL_FCM). Firstly, to reduce the computation complexity of the SR based FCM method, most energy of discriminant feature obtained by solving a SR model is reserved and the remainder is discarded. By this way, some redundant information (i.e. the correlation among samples of different classes) in the discriminant feature can be also removed, which can improve the clustering quality. Furthermore, to further enhance the clustering performance, the position information of valid values in discriminant feature is also used to re-define the distance between sample and clustering center in SL_FCM. The weighted distance in SL_FCM can enhance the similarity of the samples from the same class and the difference of the samples of different classes, thus to improve the clustering result. In addition, as the dimension of stored discriminant feature of each sample is different, we use set operations to formulate the distance and cluster center in SL_FCM. The comparisons on several datasets and images demonstrate that SL_FCM performs better than other state-of-art methods with higher accuracy, while keeps low spatial and computational complexity, especially for the large scale dataset and image.

[1]  Bor-Chen Kuo,et al.  A New Weighted Fuzzy C-Means Clustering Algorithm for Remotely Sensed Image Classification , 2011, IEEE Journal of Selected Topics in Signal Processing.

[2]  Daoqiang Zhang,et al.  Robust image segmentation using FCM with spatial constraints based on new kernel-induced distance measure , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[3]  James M. Keller,et al.  The possibilistic C-means algorithm: insights and recommendations , 1996, IEEE Trans. Fuzzy Syst..

[4]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[5]  Fang Liu,et al.  A Multi-kernel Joint Sparse Graph for SAR Image Segmentation , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[6]  James C. Bezdek,et al.  Generalized fuzzy c-means clustering strategies using Lp norm distances , 2000, IEEE Trans. Fuzzy Syst..

[7]  Xian'en Qiu,et al.  A sparse fuzzy c-means algorithm based on sparse clustering framework , 2015, Neurocomputing.

[8]  Muhammet Gulź,et al.  A state of the art literature review of VIKOR and its fuzzy extensions on applications , 2016 .

[9]  Karthikeyan Natesan Ramamurthy,et al.  Multiple Kernel Sparse Representations for Supervised and Unsupervised Learning , 2013, IEEE Transactions on Image Processing.

[10]  James M. Keller,et al.  A possibilistic fuzzy c-means clustering algorithm , 2005, IEEE Transactions on Fuzzy Systems.

[11]  Shuyuan Yang,et al.  Semi-supervised classification via kernel low-rank representation graph , 2014, Knowl. Based Syst..

[12]  Aly A. Farag,et al.  A modified fuzzy c-means algorithm for bias field estimation and segmentation of MRI data , 2002, IEEE Transactions on Medical Imaging.

[13]  Pierpaolo D'Urso,et al.  Fuzzy clustering of human activity patterns , 2013, Fuzzy Sets Syst..

[14]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[15]  Bor-Chen Kuo,et al.  A Novel Fuzzy Weighted C-Means Method for Image Classification , 2008 .

[16]  James C. Bezdek,et al.  A mixed c-means clustering model , 1997, Proceedings of 6th International Fuzzy Systems Conference.

[17]  James M. Keller,et al.  Day or Night Activity Recognition From Video Using Fuzzy Clustering Techniques , 2014, IEEE Transactions on Fuzzy Systems.

[18]  Yong Yu,et al.  Robust Subspace Segmentation by Low-Rank Representation , 2010, ICML.

[19]  Licheng Jiao,et al.  Kernel generalized fuzzy c-means clustering with spatial information for image segmentation , 2013, Digit. Signal Process..

[20]  Athanasios V. Vasilakos,et al.  Data Mining for the Internet of Things: Literature Review and Challenges , 2015, Int. J. Distributed Sens. Networks.

[21]  Hong Zhou,et al.  A modified strategy of fuzzy clustering algorithm for image segmentation , 2015, Soft Comput..

[22]  Maoguo Gong,et al.  Fuzzy Clustering With a Modified MRF Energy Function for Change Detection in Synthetic Aperture Radar Images , 2014, IEEE Transactions on Fuzzy Systems.

[23]  Zarita Zainuddin,et al.  Design of wavelet neural networks based on symmetry fuzzy C-means for function approximation , 2013, Neural Computing and Applications.

[24]  Jim Jing-Yan Wang,et al.  Discriminative sparse coding on multi-manifolds , 2013, Knowl. Based Syst..

[25]  Shuicheng Yan,et al.  Multi-task low-rank affinity pursuit for image segmentation , 2011, 2011 International Conference on Computer Vision.

[26]  Korris Fu-Lai Chung,et al.  Generalized Fuzzy C-Means Clustering Algorithm With Improved Fuzzy Partitions , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[27]  Palma Blonda,et al.  A survey of fuzzy clustering algorithms for pattern recognition. I , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[28]  R. S. Anand,et al.  A hybrid segmentation method based on Gaussian kernel fuzzy clustering and region based active contour model for ultrasound medical images , 2015, Biomed. Signal Process. Control..

[29]  Shouyi Li,et al.  Project investment decision making with fuzzy information: A literature review of methodologies based on taxonomy , 2016, J. Intell. Fuzzy Syst..

[30]  Licheng Jiao,et al.  High-Level Feature Selection With Dictionary Learning for Unsupervised SAR Imagery Terrain Classification , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[31]  Witold Pedrycz,et al.  Fuzzy C-Means clustering of incomplete data based on probabilistic information granules of missing values , 2016, Knowl. Based Syst..

[32]  S.M. Szilagyi,et al.  MR brain image segmentation using an enhanced fuzzy C-means algorithm , 2003, Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE Cat. No.03CH37439).

[33]  Rui Xu,et al.  Survey of clustering algorithms , 2005, IEEE Transactions on Neural Networks.

[34]  Licheng Jiao,et al.  Unsupervised High-Level Feature Extraction of SAR Imagery With Structured Sparsity Priors and Incremental Dictionary Learning , 2016, IEEE Geoscience and Remote Sensing Letters.

[35]  Guillermo Sapiro,et al.  Sparse Representation for Computer Vision and Pattern Recognition , 2010, Proceedings of the IEEE.

[36]  René Vidal,et al.  Sparse subspace clustering , 2009, CVPR.

[37]  James M. Keller,et al.  A possibilistic approach to clustering , 1993, IEEE Trans. Fuzzy Syst..

[38]  Xie Weixin,et al.  A novel quadrature particle filtering based on fuzzy c-means clustering , 2016 .

[39]  Horng-Lin Shieh A new framework of fuzzy clustering algorithm , 2011, 2011 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE 2011).

[40]  Zhenghua Zhou,et al.  Pose and illumination variable face recognition via sparse representation and illumination dictionary , 2016, Knowl. Based Syst..

[41]  Mohammad Hossein Fazel Zarandi,et al.  Relative entropy fuzzy c-means clustering , 2014, Inf. Sci..

[42]  Shanlin Yang,et al.  Exploring the uniform effect of FCM clustering: A data distribution perspective , 2016, Knowl. Based Syst..

[43]  Lotfi A. Zadeh,et al.  Fuzzy Sets , 1996, Inf. Control..

[44]  Fang Liu,et al.  Joint sparse regularization based Sparse Semi-Supervised Extreme Learning Machine (S3ELM) for classification , 2015, Knowl. Based Syst..

[45]  Stelios Krinidis,et al.  A Robust Fuzzy Local Information C-Means Clustering Algorithm , 2010, IEEE Transactions on Image Processing.

[46]  Hui Han,et al.  Discriminant sparse local spline embedding with application to face recognition , 2015, Knowl. Based Syst..

[47]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[48]  Q. M. Jonathan Wu,et al.  Dynamic Fuzzy Clustering and Its Application in Motion Segmentation , 2013, IEEE Transactions on Fuzzy Systems.

[49]  Ali Selamat,et al.  Systematic mapping study on granular computing , 2015, Knowl. Based Syst..

[50]  Daoqiang Zhang,et al.  Fast and robust fuzzy c-means clustering algorithms incorporating local information for image segmentation , 2007, Pattern Recognit..

[51]  Joydeep Ghosh,et al.  A study of K-Means-based algorithms for constrained clustering , 2013, Intell. Data Anal..

[52]  J. Bezdek,et al.  FCM: The fuzzy c-means clustering algorithm , 1984 .

[53]  Sven J. Dickinson,et al.  TurboPixels: Fast Superpixels Using Geometric Flows , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[54]  Ricardo J. G. B. Campello,et al.  A fuzzy extension of the Rand index and other related indexes for clustering and classification assessment , 2007, Pattern Recognit. Lett..

[55]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.