Hanson–Wright inequality in Hilbert spaces with application to $K$-means clustering for non-Euclidean data

We derive a dimensional-free Hanson-Wright inequality for quadratic forms of independent sub-gaussian random variables in a separable Hilbert space. Our inequality is an infinite-dimensional generalization of the classical Hanson-Wright inequality for finite-dimensional Euclidean random vectors. We illustrate an application to the generalized $K$-means clustering problem for non-Euclidean data. Specifically, we establish the exponential rate of convergence for a semidefinite relaxation of the generalized $K$-means, which together with a simple rounding algorithm imply the exact recovery of the true clustering structure.

[1]  G. A. Young,et al.  High‐dimensional Statistics: A Non‐asymptotic Viewpoint, Martin J.Wainwright, Cambridge University Press, 2019, xvii 552 pages, £57.99, hardback ISBN: 978‐1‐1084‐9802‐9 , 2020, International Statistical Review.

[2]  Martin J. Wainwright,et al.  High-Dimensional Statistics , 2019 .

[3]  Roman Vershynin,et al.  High-Dimensional Probability , 2018 .

[4]  Christophe Giraud,et al.  Partial recovery bounds for clustering with the relaxed K-means , 2018, Mathematical Statistics and Learning.

[5]  Yudong Chen,et al.  Hidden Integrality of SDP Relaxation for Sub-Gaussian Mixture Models , 2018, COLT.

[6]  Xiaodong Li,et al.  When do birds of a feather flock together? k-Means, proximity, and conic programming , 2017, Mathematical Programming.

[7]  Martin Royer,et al.  Adaptive Clustering through Semidefinite Programming , 2017, NIPS.

[8]  Xiaohui Chen.  Gaussian and bootstrap approximations for high-dimensional U-statistics and their applications , 2016, 1610.00032.

[9]  Florentina Bunea,et al.  PECOK: a convex optimization approach to variable clustering , 2016, 1606.05100.

[10]  T. Hsing,et al.  Theoretical foundations of functional data analysis, with an introduction to linear operators , 2015 .

[11]  慧 廣瀬.  A Mathematical Introduction to Compressive Sensing , 2015 .

[12]  R. Adamczak,et al.  A note on the Hanson-Wright inequality for random vectors with dependencies , 2014, 1409.8457.

[13]  Holger Rauhut,et al.  A Mathematical Introduction to Compressive Sensing , 2013, Applied and Numerical Harmonic Analysis.

[14]  Robin I. M. Dunbar,et al.  Do Birds of a Feather Flock Together? , 2013, Human Nature.

[15]  Van H. Vu,et al.  Random weighted projections, random quadratic forms and random eigenvectors , 2013, Random Struct. Algorithms.

[16]  M. Rudelson,et al.  Hanson-Wright inequality and sub-gaussian concentration , 2013, 1306.2872.

[17]  Francesca Ieva,et al.  Multivariate functional clustering for the morphological analysis of electrocardiograph curves , 2013 .

[18]  F. Barthe,et al.  Transference Principles for Log-Sobolev and Spectral-Gap with Applications to Conservative Spin Systems , 2012, 1202.5318.

[19]  Daniel J. Hsu,et al.  A tail inequality for quadratic forms of subgaussian random vectors , 2011, ArXiv.

[20]  Roman Vershynin,et al.  Introduction to the non-asymptotic analysis of random matrices , 2010, Compressed Sensing.

[21]  A. W. Vaart,et al.  Reproducing kernel Hilbert spaces of Gaussian priors , 2008, 0805.3252.

[22]  Le Song,et al.  A dependence maximization view of clustering , 2007, ICML '07.

[23]  Z. Q. John Lu,et al.  Nonparametric Functional Data Analysis: Theory And Practice , 2007, Technometrics.

[24]  Beyond—bernhard Schölkopf,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[25]  Thaddeus Tarpey,et al.  Clustering Functional Data , 2003, J. Classif..

[26]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[27]  F. T. Wright.  A Bound on Tail Probabilities for Quadratic Forms in Independent Random Variables Whose Distributions are not Necessarily Symmetric , 1973 .

[28]  F. T. Wright,et al.  A Bound on Tail Probabilities for Quadratic Forms in Independent Random Variables , 1971 .

[29]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .

[30]  Francesco Masulli,et al.  A survey of kernel and spectral methods for clustering , 2008, Pattern Recognit..

[31]  Jiming Peng,et al.  Advanced Optimization Laboratory Title : Approximating K-means-type clustering via semidefinite programming , 2005 .

[32]  P. Massart,et al.  ADAPTIVE ESTIMATION OF A QUADRATIC FUNCTIONAL BY MODEL SELECTION , 2001 .

[33]  R. G. Antonini.  Subgaussian random variables in Hilbert spaces , 1997 .