Variational Wasserstein Clustering

We propose a new clustering method based on optimal transportation. We discuss the connection between optimal transportation and k-means clustering, solve optimal transportation with the variational principle, and investigate the use of power diagrams as transportation plans for aggregating arbitrary domains into a fixed number of clusters. We drive cluster centroids through the target domain while maintaining the minimum clustering energy by adjusting the power diagram. Thus, we simultaneously pursue clustering and the Wasserstein distance between the centroids and the target domain, resulting in a measure-preserving mapping. We demonstrate the use of our method in domain adaptation, remeshing, and learning representations on synthetic and real data.

[1]  Bruce Fischl,et al.  FreeSurfer , 2012, NeuroImage.

[2]  Martin J. Graves,et al.  MRI from Picture to Proton , 2017 .

[3]  Leonidas J. Guibas,et al.  The Earth Mover's Distance as a Metric for Image Retrieval , 2000, International Journal of Computer Vision.

[4]  S. Graf,et al.  Foundations of Quantization for Probability Distributions , 2000 .

[5]  Shing-Tung Yau,et al.  Computational Conformal Geometry , 2016 .

[6]  Dinh Q. Phung,et al.  Multilevel Clustering via Wasserstein Means , 2017, ICML.

[7]  Jonathan Richard Shewchuk,et al.  Delaunay refinement algorithms for triangular mesh generation , 2002, Comput. Geom..

[8]  Chris H. Rycroft Voro++: a three-dimensional Voronoi cell library in C++ , 2009 .

[9]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[10]  Zhuowen Tu,et al.  Wasserstein Introspective Neural Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[11]  C. Givens,et al.  A class of Wasserstein metrics for probability distributions. , 1984 .

[12]  Simon Urbanek,et al.  Unsupervised clustering of multidimensional distributions using earth mover distance , 2011, KDD.

[13]  Mathieu Desbrun,et al.  Weighted Triangulations for Geometry Processing , 2014, ACM Trans. Graph..

[14]  E. Forgy,et al.  Cluster analysis of multivariate data : efficiency versus interpretability of classifications , 1965 .

[15]  Wei Zeng,et al.  Optimal Mass Transport for Shape Matching and Comparison , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Arthur Cayley,et al.  The Collected Mathematical Papers: On Monge's “Mémoire sur la théorie des déblais et des remblais” , 2009 .

[17]  Chris H Rycroft,et al.  VORO++: a three-dimensional voronoi cell library in C++. , 2009, Chaos.

[18]  Pierre Alliez,et al.  CGAL - The Computational Geometry Algorithms Library , 2011 .

[19]  Alison L Gibbs,et al.  On Choosing and Bounding Probability Metrics , 2002, math/0209021.

[20]  Y. Brenier Polar Factorization and Monotone Rearrangement of Vector-Valued Functions , 1991 .

[21]  Hans-Peter Seidel,et al.  Animating deformable objects using sparse spacetime constraints , 2014, ACM Trans. Graph..

[22]  James Zijun Wang,et al.  Fast Discrete Distribution Clustering Using Wasserstein Barycenter With Sparse Support , 2015, IEEE Transactions on Signal Processing.

[23]  Guillaume Carlier,et al.  Barycenters in the Wasserstein Space , 2011, SIAM J. Math. Anal..

[24]  Marco Cuturi,et al.  Sinkhorn Distances: Lightspeed Computation of Optimal Transport , 2013, NIPS.

[25]  Nick C Fox,et al.  Brain atrophy progression measured from registered serial MRI: Validation and application to alzheimer's disease , 1997, Journal of magnetic resonance imaging : JMRI.

[26]  Haibin Ling,et al.  An Efficient Earth Mover's Distance Algorithm for Robust Histogram Comparison , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Paul M. Thompson,et al.  Volumetric harmonic brain mapping , 2004, 2004 2nd IEEE International Symposium on Biomedical Imaging: Nano to Macro (IEEE Cat No. 04EX821).

[28]  C. Villani Topics in Optimal Transportation , 2003 .

[29]  Léon Bottou,et al.  Wasserstein Generative Adversarial Networks , 2017, ICML.

[30]  Nicolas Courty,et al.  Domain Adaptation with Regularized Optimal Transport , 2014, ECML/PKDD.

[31]  Quentin Mérigot,et al.  A Multiscale Approach to Optimal Transport , 2011, Comput. Graph. Forum.

[32]  B. Lévy A Numerical Algorithm for L2 Semi-Discrete Optimal Transport in 3D , 2014, 1409.1279.

[33]  Arnaud Doucet,et al.  Fast Computation of Wasserstein Barycenters , 2013, ICML.

[34]  Sergei Vassilvitskii,et al.  k-means++: the advantages of careful seeding , 2007, SODA '07.

[35]  Hang Si,et al.  TetGen, a Delaunay-Based Quality Tetrahedral Mesh Generator , 2015, ACM Trans. Math. Softw..

[36]  Hossein Mobahi,et al.  Learning with a Wasserstein Loss , 2015, NIPS.

[37]  Nicolas Courty,et al.  Joint distribution optimal transportation for domain adaptation , 2017, NIPS.

[38]  S. Yau,et al.  Variational Principles for Minkowski Type Problems, Discrete Optimal Transport, and Discrete Monge-Ampere Equations , 2013, 1302.5472.

[39]  Franz Aurenhammer,et al.  Power Diagrams: Properties, Algorithms and Applications , 1987, SIAM J. Comput..

[40]  L. Kantorovich On the Translocation of Masses , 2006 .

[41]  D. Chand,et al.  On Convex Polyhedra , 1970 .