Unsupervised Kernel Learning for Correlation Based Clustering

Successful clustering of multiple objects using kernels, heavily relies on the proper selection of kernel parameters. This can be a computationally complex process and may necessitate prior knowledge of label information. In this paper, a novel method has been introduced that is computationally efficient and requires no prior information. The method relies on the eigenvalues of each kernel matrix to determine a proper linear combination of kernels among a dictionary of kernels that results in good clustering. A difference of convex functions formulation is proposed and solved via an algorithmically simple method which is extremely cost-effective in implementation. Comparisons using various forms of real-world data with two popular supervised methods show its superior performance.

[1]  N. Cristianini,et al.  On Kernel-Target Alignment , 2001, NIPS.

[2]  Ioannis D. Schizas,et al.  Fault tolerant unsupervised kernel-based information clustering in hyperspectral images , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[3]  Dale Schuurmans,et al.  Maximum Margin Clustering , 2004, NIPS.

[4]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[5]  Steven C. H. Hoi,et al.  Unsupervised Multiple Kernel Learning , 2011, ACML.

[6]  Mehryar Mohri,et al.  Algorithms for Learning Kernels Based on Centered Alignment , 2012, J. Mach. Learn. Res..

[7]  Jia Chen,et al.  Online Distributed Sparsity-Aware Canonical Correlation Analysis , 2016, IEEE Transactions on Signal Processing.

[8]  Michael L. Overton,et al.  Optimality conditions and duality theory for minimizing sums of the largest eigenvalues of symmetric matrices , 2015, Math. Program..

[9]  Ye Zhang,et al.  Representative Multiple Kernel Learning for Classification in Hyperspectral Imagery , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[10]  Ioannis D. Schizas,et al.  Unsupervised Kernel Correlations Based Hyperspectral Clustering With Missing Pixels , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[11]  Ethem Alpaydin,et al.  Multiple Kernel Learning Algorithms , 2011, J. Mach. Learn. Res..

[12]  Rong Jin,et al.  Generalized Maximum Margin Clustering and Unsupervised Kernel Learning , 2006, NIPS.

[13]  Daoqiang Zhang,et al.  Adaptive Kernel Principal Component Analysis with Unsupervised Learning of Kernels , 2006, Sixth International Conference on Data Mining (ICDM'06).

[14]  Jia Chen,et al.  Data-driven sensors clustering and filtering for communication efficient field reconstruction , 2017, Signal Process..

[15]  Georgios B. Giannakis,et al.  Online Ensemble Multi-kernel Learning Adaptive to Non-stationary and Adversarial Environments , 2017, AISTATS.

[16]  Alexander J. Smola,et al.  Learning with Kernels: support vector machines, regularization, optimization, and beyond , 2001, Adaptive computation and machine learning series.

[17]  T. P. Dinh,et al.  Convex analysis approach to d.c. programming: Theory, Algorithm and Applications , 1997 .

[18]  Daniela Micucci,et al.  UniMiB SHAR: a new dataset for human activity recognition using acceleration data from smartphones , 2016, ArXiv.