Ramp-based twin support vector clustering

Traditional plane-based clustering methods measure the within-cluster or between-cluster scatter by linear, quadratic or some other unbounded functions, which are sensitive to the samples far from the cluster center. This paper introduces the ramp functions into plane-based clustering and proposes a ramp-based twin support vector clustering (RampTWSVC). RampTWSVC is very robust to the samples far from the cluster center, because its within-cluster and between-cluster scatters are measured by the bounded ramp functions. Thus, it is easier to find the intrinsic clusters than other plane-based clustering methods. The nonconvex programming problem in RampTWSVC is solved efficiently through an alternating iteration algorithm, and its local solution can be obtained in a finite number of iterations theoretically. In addition, its nonlinear manifold clustering formation is also proposed via a kernel trick. Experimental results on several benchmark datasets show the better performance of our RampTWSVC compared with other plane-based clustering methods.

[1]  Jason Weston,et al.  Large Scale Transductive SVMs , 2006, J. Mach. Learn. Res..

[2]  Yuan-Hai Shao,et al.  Improvements on Twin Support Vector Machines , 2011, IEEE Transactions on Neural Networks.

[3]  Paul S. Bradley,et al.  k-Plane Clustering , 2000, J. Glob. Optim..

[4]  Reshma Khemchandani,et al.  Fuzzy least squares twin support vector clustering , 2016, Neural Computing and Applications.

[5]  Seyed Mojtaba Hosseini Bamakan,et al.  Ramp loss one-class support vector machine; A robust and effective approach to anomaly detection problems , 2018, Neurocomputing.

[6]  Yong Shi,et al.  Ramp loss least squares support vector machine , 2016, J. Comput. Sci..

[7]  Robert M. Haralick,et al.  Nonlinear Manifold Clustering By Dimensionality , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[8]  Robert Pless,et al.  Manifold clustering , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[9]  Olvi L. Mangasarian,et al.  Multisurface proximal support vector machine classification via generalized eigenvalues , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Yuan-Hai Shao,et al.  Robust and Sparse Linear Discriminant Analysis via an Alternating Direction Method of Multipliers , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[11]  Michael W. Berry,et al.  Survey of Text Mining: Clustering, Classification, and Retrieval , 2007 .

[12]  Yuan Zhang,et al.  Fuzzy clustering with the entropy of attribute weights , 2016, Neurocomputing.

[13]  Anil K. Jain,et al.  Algorithms for Clustering Data , 1988 .

[14]  Vipin Kumar,et al.  Introduction to Data Mining, (First Edition) , 2005 .

[15]  Lan Bai,et al.  Proximal Plane Clustering via Eigenvalues , 2013, ITQM.

[16]  Roman Ilin,et al.  Unsupervised Learning of Categorical Data With Competing Models , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[17]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[18]  Daniel T. Larose,et al.  Discovering Knowledge in Data: An Introduction to Data Mining , 2005 .

[19]  Zhao Kang,et al.  Robust Subspace Clustering via Smoothed Rank Approximation , 2015, IEEE Signal Processing Letters.

[20]  Lan Bai,et al.  Clustering by twin support vector machine and least square twin support vector classifier with uniform output coding , 2019, Knowl. Based Syst..

[21]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[22]  Rongfang Bie,et al.  Clustering by fast search and find of density peaks via heat diffusion , 2016, Neurocomputing.

[23]  Yuan-Hai Shao,et al.  k-Proximal plane clustering , 2017, Int. J. Mach. Learn. Cybern..

[24]  Michael W. Berry,et al.  Survey of Text Mining , 2003, Springer New York.

[25]  Ioannis D. Schizas,et al.  MILP-Based Unsupervised Clustering , 2018, IEEE Signal Processing Letters.

[26]  Zechao Li,et al.  L1-Norm Distance Minimization-Based Fast Robust Twin Support Vector $k$ -Plane Clustering , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[27]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Hakan Cevikalp,et al.  Best Fitting Hyperplanes for Classification , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Yuan-Hai Shao,et al.  A GA-based model selection for smooth twin parametric-margin support vector machine , 2013, Pattern Recognit..

[30]  Zhi-Hua Zhou,et al.  A brief introduction to weakly supervised learning , 2018 .

[31]  S. Logeswari,et al.  A Survey on Text Mining in Clustering , 2011 .

[32]  Lan Bai,et al.  Stochastic Gradient Twin Support Vector Machine for Large Scale Problems , 2017, Inf. Sci..

[33]  Yun Fu,et al.  Clustering with Partition Level Side Information , 2015, 2015 IEEE International Conference on Data Mining.

[34]  Tommy W. S. Chow,et al.  Organizing Books and Authors by Multilayer SOM , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[35]  Yunming Ye,et al.  Extensions of Kmeans-Type Algorithms: A New Clustering Framework by Integrating Intracluster Compactness and Intercluster Separation , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[36]  Reshma Khemchandani,et al.  Optimal kernel selection in twin support vector machines , 2009, Optim. Lett..

[37]  Lan Bai,et al.  Twin Support Vector Machine for Clustering , 2015, IEEE Transactions on Neural Networks and Learning Systems.