A Semi-Definite Programming approach to low dimensional embedding for unsupervised clustering

This paper proposes a variant of the method of Gu\'edon and Verhynin for estimating the cluster matrix in the Mixture of Gaussians framework via Semi-Definite Programming. A clustering oriented embedding is deduced from this estimate. The procedure is suitable for very high dimensional data because it is based on pairwise distances only. Theoretical garantees are provided and an eigenvalue optimisation approach is proposed for computing the embedding. The performance of the method is illustrated via Monte Carlo experiements and comparisons with other embeddings from the literature.

[1]  R. Samworth,et al.  Random‐projection ensemble classification , 2015, 1504.04595.

[2]  H. Akaike.  A new look at the statistical model identification , 1974 .

[3]  Franz Rendl,et al.  A Spectral Bundle Method for Semidefinite Programming , 1999, SIAM J. Optim..

[4]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[5]  Anil K. Jain.  Data clustering: 50 years beyond K-means , 2010, Pattern Recognit. Lett..

[6]  Gérard Govaert,et al.  Assessing a Mixture Model for Clustering with the Integrated Completed Likelihood , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Emmanuel Abbe,et al.  Exact Recovery in the Stochastic Block Model , 2014, IEEE Transactions on Information Theory.

[8]  W. B. Johnson,et al.  Extensions of Lipschitz mappings into Hilbert space , 1984 .

[9]  Laurent Massoulié,et al.  Community Detection in the Labelled Stochastic Block Model , 2012, ArXiv.

[10]  Francis R. Bach,et al.  Clusterpath: an Algorithm for Clustering using Convex Fusion Penalties , 2011, ICML.

[11]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[12]  P. Deb.  Finite Mixture Models , 2008 .

[13]  P. Radchenko,et al.  Consistent clustering using l 1 fusion penalty , 2014 .

[14]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[15]  C. Biernacki,et al.  Degeneracy in the maximum likelihood estimation of univariate Gaussian mixtures with EM , 2003 .

[16]  Van H. Vu.  Singular vectors under random perturbation , 2011, Random Struct. Algorithms.

[17]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[18]  Gábor Lugosi,et al.  Concentration Inequalities - A Nonasymptotic Theory of Independence , 2013, Concentration Inequalities.

[19]  Gary K. Chen,et al.  Convex Clustering: An Attractive Alternative to Hierarchical Clustering , 2014, PLoS Comput. Biol..

[20]  Wei Sun,et al.  Sparse Convex Clustering , 2016, ArXiv.

[21]  Joel A. Tropp,et al.  Column subset selection, matrix factorization, and eigenvalue optimization , 2008, SODA.

[22]  Luca Trevisan,et al.  Multi-way spectral partitioning and higher-order cheeger inequalities , 2011, STOC '12.

[23]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[24]  Roman Vershynin,et al.  Community detection in sparse networks via Grothendieck’s inequality , 2014, Probability Theory and Related Fields.

[25]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[26]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[27]  G. Celeux,et al.  A Classification EM algorithm for clustering and two stochastic versions , 1992 .

[28]  Kean Ming Tan,et al.  Statistical properties of convex clustering. , 2015, Electronic journal of statistics.

[29]  Kilian Q. Weinberger,et al.  Unsupervised Learning of Image Manifolds by Semidefinite Programming , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..