Geometry and clustering with metrics derived from separable Bregman divergences

Separable Bregman divergences induce Riemannian metric spaces that are isometric to the Euclidean space after monotone embeddings. We investigate fixed rate quantization and its codebook Voronoi diagrams, and report on experimental performances of partition-based, hierarchical, and soft clustering algorithms with respect to these Riemann-Bregman distances.

[1]  Richard Nock,et al.  On Bregman Voronoi diagrams , 2007, SODA '07.

[2]  Inderjit S. Dhillon,et al.  A generalized maximum entropy approach to bregman co-clustering and matrix approximation , 2004, J. Mach. Learn. Res..

[3]  S. Amari,et al.  Information geometry of divergence functions , 2010 .

[4]  Frank Nielsen,et al.  An Elementary Introduction to Information Geometry , 2018, Entropy.

[5]  L. Bregman.  The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming , 1967 .

[6]  Inderjit S. Dhillon,et al.  Clustering with Bregman Divergences , 2005, J. Mach. Learn. Res..

[7]  Aurélie Fischer,et al.  Quantization and clustering with Bregman divergences , 2010, J. Multivar. Anal..

[8]  Heinz H. Bauschke,et al.  Construction of best Bregman approximations in reflexive Banach spaces , 2003 .

[9]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[10]  Chong Li,et al.  The Bregman distance, approximate compactness and convexity of Chebyshev sets in Banach spaces , 2010, J. Approx. Theory.

[11]  Heinz H. Bauschke,et al.  Bregman Monotone Optimization Algorithms , 2003, SIAM J. Control. Optim..

[12]  A. Schwartzman.  Lognormal Distributions and Geometric Averages of Positive Definite Matrices , 2014, 1407.6383.

[13]  Heinz H. Bauschke,et al.  Dykstras algorithm with bregman projections: A convergence proof , 2000 .

[14]  Dan Butnariu,et al.  Bregman distances, totally convex functions, and a method for solving operator equations in Banach spaces , 2006 .

[15]  C. Udriste,et al.  Geometric Modeling in Probability and Statistics , 2014 .

[16]  Henryk Gzyl,et al.  Prediction in Riemannian metrics derived from divergence functions , 2020, Communications in Statistics - Theory and Methods.

[17]  D. Pollard.  A User's Guide to Measure Theoretic Probability by David Pollard , 2001 .

[18]  T. Linder.  LEARNING-THEORETIC METHODS IN VECTOR QUANTIZATION , 2002 .

[19]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[20]  Xin Guo,et al.  On the optimality of conditional expectation as a Bregman predictor , 2005, IEEE Trans. Inf. Theory.

[21]  Yair Censor,et al.  The Dykstra algorithm with Bregman projec-tions , 1998 .

[22]  Jimmie D. Lawson,et al.  The Geometric Mean, Matrices, Metrics, and More , 2001, Am. Math. Mon..

[23]  Rodney W. Johnson,et al.  Axiomatic Characterization of A Family of Information Measures That Contains the Directed Divergences. , 1977 .

[24]  Frank Nielsen,et al.  Sided and Symmetrized Bregman Centroids , 2009, IEEE Transactions on Information Theory.

[25]  S. Lang,et al.  Math Talks for Undergraduates , 1999 .

[26]  Yair Censor,et al.  Algorithms and Convergence Results of Projection Methods for Inconsistent Feasibility Problems: A Review , 2018, 1802.07529.

[27]  Heinz H. Bauschke,et al.  Legendre functions and the method of random Bregman projections , 1997 .

[28]  Frank Nielsen,et al.  Introduction to HPC with MPI for Data Science , 2016, Undergraduate Topics in Computer Science.

[29]  Julie Josse,et al.  Principal component methods - hierarchical clustering - partitional clustering: why would we need to choose for visualizing data? , 2010 .

[30]  Maher Moakher,et al.  A Differential Geometric Approach to the Geometric Mean of Symmetric Positive-Definite Matrices , 2005, SIAM J. Matrix Anal. Appl..