Convex clustering with metric learning

Abstract The convex clustering formulation of Chi and Lange (2015) is revisited. While this formulation can be precisely and efficiently solved, it uses the standard Euclidean metric to measure the distance between the data points and their corresponding cluster centers and hence its performance deteriorates significantly in the presence of outlier features. To address this issue, this paper considers a formulation that combines convex clustering with metric learning. It is shown that: (1) for any given positive definite Mahalanobis distance metric, the problem of convex clustering can be precisely and efficiently solved using the Alternating Direction Method of Multipliers; (2) the problem of learning a positive definite Mahalanobis distance metric admits a closed-form solution; (3) an algorithm that alternates between convex clustering and metric learning can provide a significant performance boost over not only the original convex clustering formulation but also the recently proposed robust convex clustering formulation of Wang et al. (2017).

[1]  Fei Yin,et al.  Handwritten Chinese text line segmentation by clustering with distance metric learning , 2009, Pattern Recognit..

[2]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[3]  Hong Chang,et al.  Semi-supervised distance metric learning , 2006 .

[4]  R. Glowinski,et al.  Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de Dirichlet non linéaires , 1975 .

[5]  Å. Björck.  Numerical Methods in Matrix Computations , 2014 .

[6]  Xin Jin,et al.  K-Means Clustering , 2010, Encyclopedia of Machine Learning.

[7]  Michael I. Jordan,et al.  Learning Spectral Clustering , 2003, NIPS.

[8]  Longin Jan Latecki,et al.  Distance Learning Based on Convex Clustering , 2009, ISVC.

[9]  Francis R. Bach,et al.  Clusterpath: an Algorithm for Clustering using Convex Fusion Penalties , 2011, ICML.

[10]  Fei Wang,et al.  Survey on distance metric learning and dimensionality reduction in data mining , 2014, Data Mining and Knowledge Discovery.

[11]  Eric C. Chi,et al.  Splitting Methods for Convex Clustering , 2013, Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America.

[12]  Ming Shao,et al.  Infinite Ensemble for Image Clustering , 2016, KDD.

[13]  Camille Roth,et al.  Natural Scales in Geographical Patterns , 2017, Scientific Reports.

[14]  Fei Wang,et al.  Improving clustering by learning a bi-stochastic data similarity matrix , 2011, Knowledge and Information Systems.

[15]  Xuelong Li,et al.  Feedback mechanism based iterative metric learning for person re-identification , 2018, Pattern Recognit..

[16]  Richard H. Bartels,et al.  Algorithm 432 [C2]: Solution of the matrix equation AX + XB = C [F4] , 1972, Commun. ACM.

[17]  Bernard De Baets,et al.  Supervised distance metric learning through maximization of the Jeffrey divergence , 2017, Pattern Recognit..

[18]  Peter Bühlmann,et al.  Supervised clustering of genes , 2002, Genome Biology.

[19]  Xenophon Papademetris,et al.  Groupwise whole-brain parcellation from resting-state fMRI data for network node identification , 2013, NeuroImage.

[20]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[21]  Chin-Chun Chang,et al.  A boosting approach for supervised Mahalanobis distance metric learning , 2012, Pattern Recognit..

[22]  William M. Rand,et al.  Objective Criteria for the Evaluation of Clustering Methods , 1971 .

[23]  Jiayu Zhou,et al.  Robust Convex Clustering Analysis , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[24]  Antony Jameson,et al.  SOLUTION OF EQUATION AX + XB = C BY INVERSION OF AN M × M OR N × N MATRIX ∗ , 1968 .

[25]  George M. Siouris,et al.  Applied Optimal Control: Optimization, Estimation, and Control , 1979, IEEE Transactions on Systems, Man, and Cybernetics.

[26]  Qinghua Hu,et al.  Subspace clustering guided unsupervised feature selection , 2017, Pattern Recognit..

[27]  L. Ljung,et al.  Just Relax and Come Clustering! : A Convexification of k-Means Clustering , 2011 .

[28]  D. Gabay.  Applications of the method of multipliers to variational inequalities , 1983 .

[29]  Yaoliang Yu,et al.  Distance metric learning by minimal distance maximization , 2011, Pattern Recognit..

[30]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[31]  Feiping Nie,et al.  Learning a Mahalanobis distance metric for data clustering and classification , 2008, Pattern Recognit..

[32]  Daoqiang Zhang,et al.  Semi-supervised clustering with metric learning: An adaptive kernel method , 2010, Pattern Recognit..

[33]  Jieping Ye,et al.  Adaptive Distance Metric Learning for Clustering , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Antony Jameson,et al.  Solution of the Equation $AX + XB = C$ by Inversion of an $M \times M$ or $N \times N$ Matrix , 1968 .

[35]  Francis R. Bach,et al.  Large-Margin Metric Learning for Constrained Partitioning Problems , 2014, ICML.

[36]  Michael I. Jordan,et al.  Distance Metric Learning with Application to Clustering with Side-Information , 2002, NIPS.

[37]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..