Non-flat clustering with alpha-divergences

The scope of the well-known k-means algorithm has been broadly extended with some recent results: first, the k-means++ initialization method gives some approximation guarantees; second, the Bregman k-means algorithm generalizes the classical algorithm to the large family of Bregman divergences. The Bregman seeding framework combines approximation guarantees with Bregman divergences. We present here an extension of the k-means algorithm using the family of α-divergences. With the framework for representational Bregman divergences, we show that an α-divergence based k-means algorithm can be designed. We present preliminary experiments for clustering and image segmentation applications. Since α-divergences are the natural divergences for constant curvature spaces, these experiments are expected to give information on the structure of the data.

[1]  Imre Csiszár,et al.  Axiomatic Characterizations of Information Measures , 2008, Entropy.

[2]  Leszek Gasieniec,et al.  Proceedings of the eighteenth annual ACM-SIAM symposium on discrete algorithms , 2007, SODA 2007.

[3]  N. Čencov Statistical Decision Rules and Optimal Inference , 2000 .

[4]  Frank Nielsen,et al.  Bregman Voronoi Diagrams , 2007, Discret. Comput. Geom..

[5]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[6]  Shun-ichi Amari,et al.  Methods of information geometry , 2000 .

[7]  Antonio Torralba,et al.  Building the gist of a scene: the role of global image features in recognition. , 2006, Progress in brain research.

[8]  Frank Nielsen,et al.  The Dual Voronoi Diagrams with Respect to Representational Bregman Divergences , 2009, 2009 Sixth International Symposium on Voronoi Diagrams.

[9]  Richard Nock,et al.  Mixed Bregman Clustering with Approximation Guarantees , 2008, ECML/PKDD.

[10]  Sergei Vassilvitskii,et al.  k-means++: the advantages of careful seeding , 2007, SODA '07.

[11]  Inderjit S. Dhillon,et al.  Clustering with Bregman Divergences , 2005, J. Mach. Learn. Res..

[12]  Mihoko Minami,et al.  Robust Blind Source Separation by Beta Divergence , 2002, Neural Computation.

[13]  Joydeep Ghosh,et al.  Cluster Ensembles --- A Knowledge Reuse Framework for Combining Multiple Partitions , 2002, J. Mach. Learn. Res..

[14]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[15]  Inderjit S. Dhillon,et al.  Concept Decompositions for Large Sparse Text Data Using Clustering , 2004, Machine Learning.