Weighted Cheeger and Buser Inequalities, with Applications to Clustering and Cutting Probability Densities

In this paper, we show how sparse or isoperimetric cuts of a probability density function relate to Cheeger cuts of its principal eigenfunction, for appropriate definitions of `sparse cut' and `principal eigenfunction'. We construct these appropriate definitions of sparse cut and principal eigenfunction in the probability density setting. Then, we prove Cheeger and Buser type inequalities similar to those for the normalized graph Laplacian of Alon-Milman. We demonstrate that no such inequalities hold for most prior definitions of sparse cut and principal eigenfunction. We apply this result to generate novel algorithms for cutting probability densities and clustering data, including a principled variant of spectral clustering.

[1]  Ankur Moitra,et al.  Settling the Polynomial Learnability of Mixtures of Gaussians , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[2]  Miklós Simonovits,et al.  The mixing rate of Markov chains, an isoperimetric inequality, and computing the volume , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[3]  Shang-Hua Teng,et al.  Nearly-linear time algorithms for graph partitioning, graph sparsification, and solving linear systems , 2003, STOC '04.

[4]  M. Fiedler Algebraic connectivity of graphs , 1973 .

[5]  Tugkan Batu,et al.  Generalized Uniformity Testing , 2017, 2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS).

[6]  Jitendra Malik,et al.  Normalized Cuts and Image Segmentation , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  S. Yau,et al.  Surveys in Differential Geometry , 1999 .

[8]  Aleksander Madry,et al.  Fast Approximation Algorithms for Cut-Based Problems in Undirected Graphs , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[9]  David P. Woodruff,et al.  Strong Coresets for k-Median and Subspace Approximation: Goodbye Dimension , 2018, 2018 IEEE 59th Annual Symposium on Foundations of Computer Science (FOCS).

[10]  E. Milman On the role of convexity in isoperimetry, spectral gap and concentration , 2007, 0712.4092.

[11]  Daniel M. Kane,et al.  Bounded Independence Fools Degree-2 Threshold Functions , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[12]  M. Gromov,et al.  A topological application of the isoperimetric inequality , 1983 .

[13]  Di Wang,et al.  Expander Decomposition and Pruning: Faster, Stronger, and Simpler , 2018, SODA.

[14]  D. Spielman,et al.  Spectral partitioning works: planar graphs and finite element meshes , 1996, Proceedings of 37th Conference on Foundations of Computer Science.

[15]  Miklós Simonovits,et al.  Isoperimetric problems for convex bodies and a localization lemma , 1995, Discret. Comput. Geom..

[16]  Andrej Risteski,et al.  Beyond Log-concavity: Provable Guarantees for Sampling Multi-modal Distributions using Simulated Tempering Langevin Monte Carlo , 2017, NeurIPS.

[17]  Yuval Peres,et al.  Finding sparse cuts locally using evolving sets , 2008, STOC '09.

[18]  K. Friedrichs The identity of weak and strong extensions of differential operators , 1944 .

[19]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[20]  Mikhail Belkin,et al.  Semi-Supervised Learning on Riemannian Manifolds , 2004, Machine Learning.

[21]  Nisheeth K. Vishnoi,et al.  Towards an SDP-based approach to spectral methods: a nearly-linear-time algorithm for graph partitioning and decomposition , 2010, SODA '11.

[22]  Noga Alon,et al.  Eigenvalues, Expanders and Superconcentrators (Extended Abstract) , 1984, FOCS.

[23]  J. Cheeger A lower bound for the smallest eigenvalue of the Laplacian , 1969 .

[24]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[25]  Leo Grady,et al.  Isoperimetric graph partitioning for image segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Nisheeth K. Vishnoi,et al.  On partitioning graphs via single commodity flows , 2008, STOC.

[27]  Anupam Gupta,et al.  Embeddings of negative-type metrics and an improved approximation to generalized sparsest cut , 2005, SODA '05.

[28]  Frank Thomson Leighton,et al.  An approximate max-flow min-cut theorem for uniform multicommodity flow problems with applications to approximation algorithms , 1988, [Proceedings 1988] 29th Annual Symposium on Foundations of Computer Science.

[29]  Prasad Raghavendra,et al.  Many sparse cuts via higher eigenvalues , 2011, STOC '12.

[30]  Luca Trevisan,et al.  Multi-way spectral partitioning and higher-order cheeger inequalities , 2011, STOC '12.

[31]  Mikhail Belkin,et al.  Consistency of spectral clustering , 2008, 0804.0678.

[32]  Santosh S. Vempala,et al.  Stochastic localization + Stieltjes barrier = tight bound for log-Sobolev , 2018, STOC.

[33]  Gary L. Miller,et al.  Hardy-Muckenhoupt Bounds for Laplacian Eigenvalues , 2018, APPROX-RANDOM.

[34]  Robert Krauthgamer,et al.  Cheeger-Type Approximation for Sparsest st-Cut , 2014, ACM Trans. Algorithms.

[35]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[36]  P. Buser A note on the isoperimetric constant , 1982 .

[37]  Santosh S. Vempala,et al.  The Kannan-Lovász-Simonovits Conjecture , 2018, ArXiv.

[38]  Mikhail Belkin,et al.  On Learning with Integral Operators , 2010, J. Mach. Learn. Res..

[39]  E. Stein,et al.  Real Analysis: Measure Theory, Integration, and Hilbert Spaces , 2005 .

[40]  Shang-Hua Teng,et al.  Spectral partitioning works: planar graphs and finite element meshes , 1996, Proceedings of 37th Conference on Foundations of Computer Science.

[41]  Amit Kumar,et al.  Clustering with Spectral Norm and the k-Means Algorithm , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[42]  Stephen P. Boyd,et al.  A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights , 2014, J. Mach. Learn. Res..

[43]  Satish Rao,et al.  Expander flows, geometric embeddings and graph partitioning , 2004, STOC '04.

[44]  Andre Wibisono,et al.  A variational perspective on accelerated methods in optimization , 2016, Proceedings of the National Academy of Sciences.

[45]  Martin E. Dyer,et al.  A random polynomial-time algorithm for approximating the volume of convex bodies , 1991, JACM.

[46]  Christian Wulff-Nilsen,et al.  Fully-dynamic minimum spanning forest with improved worst-case update time , 2016, STOC.

[47]  B. Muckenhoupt Hardy's inequality with weights , 1972 .