A Game-Theoretic Approach to Hypergraph Clustering

Hypergraph clustering refers to the process of extracting maximally coherent groups from a set of objects using high-order (rather than pairwise) similarities. Traditional approaches to this problem are based on the idea of partitioning the input data into a predetermined number of classes, thereby obtaining the clusters as a by-product of the partitioning process. In this paper, we offer a radically different view of the problem. In contrast to the classical approach, we attempt to provide a meaningful formalization of the very notion of a cluster and we show that game theory offers an attractive and unexplored perspective that serves our purpose well. To this end, we formulate the hypergraph clustering problem in terms of a noncooperative multiplayer “clustering game,” and show that a natural notion of a cluster turns out to be equivalent to a classical (evolutionary) game-theoretic equilibrium concept. We prove that the problem of finding the equilibria of our clustering game is equivalent to locally optimizing a polynomial function over the standard simplex, and we provide a discrete-time high-order replicator dynamics to perform this optimization, based on the Baum-Eagon inequality. Experiments over synthetic as well as real-world data are presented which show the superiority of our approach over the state of the art.

[1]  Tamir Hazan,et al.  Multi-way Clustering Using Super-Symmetric Non-negative Tensor Factorization , 2006, ECCV.

[2]  Josef Hofbauer,et al.  Evolutionary Games and Population Dynamics , 1998 .

[3]  Katherine A. Heller,et al.  A Nonparametric Bayesian Approach to Modeling Overlapping Clusters , 2007, AISTATS.

[4]  David J. Kriegman,et al.  What Is the Set of Images of an Object Under All Possible Illumination Conditions? , 1998, International Journal of Computer Vision.

[5]  L. Baum,et al.  Growth transformations for functions on manifolds. , 1968 .

[6]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Bernhard Schölkopf,et al.  Learning with Hypergraphs: Clustering, Classification, and Embedding , 2006, NIPS.

[8]  Jon M. Kleinberg,et al.  Clustering categorical data: an approach based on dynamical systems , 2000, The VLDB Journal.

[9]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using unit quaternions , 1987 .

[10]  Olivier D. Faugeras,et al.  Improving Consistency and Reducing Ambiguity in Stochastic Labeling: An Optimization Approach , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  J M Smith,et al.  Evolution and the theory of games , 1976 .

[12]  Claude Berge,et al.  Hypergraphs - combinatorics of finite sets , 1989, North-Holland mathematical library.

[13]  J. Rodríguez.  On the Laplacian Spectrum and Walk-regular Hypergraphs , 2003 .

[14]  Vipin Kumar,et al.  Multilevel k-way hypergraph partitioning , 1999, DAC '99.

[15]  Marianna Bolla,et al.  Spectra, Euclidean representations and clusterings of hypergraphs , 1993, Discret. Math..

[16]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Martine D. F. Schlag,et al.  Multi-level spectral hypergraph partitioning with arbitrary vertex sizes , 1996, Proceedings of International Conference on Computer Aided Design.

[18]  Koby Crammer,et al.  A rate-distortion one-class model and its applications to clustering , 2008, ICML '08.

[19]  Joydeep Ghosh,et al.  Robust one-class clustering using hybrid global and local search , 2005, ICML.

[20]  Robin Sibson,et al.  The Construction of Hierarchic and Non-Hierarchic Classifications , 1968, Comput. J..

[21]  and Charles K. Taft Reswick,et al.  Introduction to Dynamic Systems , 1967 .

[22]  Shuicheng Yan,et al.  Robust Clustering as Ensembles of Affinity Relations , 2010, NIPS.

[23]  G. R. Blakley.  Homogeneous nonnegative symmetric quadratic transformations , 1964 .

[24]  Shimon Ullman,et al.  Structural Saliency: The Detection Of Globally Salient Structures using A Locally Connected Network , 1988, [1988 Proceedings] Second International Conference on Computer Vision.

[25]  Venu Madhav Govindu,et al.  A tensor decomposition for geometric grouping and segmentation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[26]  Joydeep Ghosh,et al.  Model-based overlapping clustering , 2005, KDD '05.

[27]  Andrea Torsello,et al.  Grouping with Asymmetric Affinities: A Game-Theoretic Perspective , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[28]  Andrea Torsello,et al.  Payoff-Monotonic Game Dynamics and the Maximum Clique Problem , 2006, Neural Computation.

[29]  Samuel,et al.  A Game-Theoretic Framework for Similarity-Based Data Clustering , 2009 .

[30]  Marcello Pelillo,et al.  Graph-based quadratic optimization: A fast evolutionary approach , 2011, Comput. Vis. Image Underst..

[31]  Marcello Pelillo,et al.  Dominant Sets and Pairwise Clustering , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Kim L. Boyer,et al.  Quantitative measures of change based on feature organization: eigenvalues and eigenvectors , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[33]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[34]  Yihong Gong,et al.  Unsupervised Image Categorization by Hypergraph Partition , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Pietro Perona,et al.  Beyond pairwise clustering , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[36]  L. Baum,et al.  An inequality with applications to statistical estimation for probabilistic functions of Markov processes and to a model for ecology , 1967 .

[37]  Noam Nisan,et al.  A Note on the computational hardness of evolutionary stable strategies , 2006, Electron. Colloquium Comput. Complex..

[38]  C Cannings,et al.  Multi-player matrix games. , 1997, Bulletin of mathematical biology.

[39]  Marcello Pelillo,et al.  The Dynamics of Nonlinear Relaxation Labeling Processes , 1997, Journal of Mathematical Imaging and Vision.

[40]  Vipin Kumar,et al.  Multilevel k-way Hypergraph Partitioning , 2000, VLSI Design.

[41]  Serge J. Belongie,et al.  Higher order learning with graphs , 2006, ICML.

[42]  Kousha Etessami,et al.  The computational complexity of evolutionarily stable strategies , 2008, Int. J. Game Theory.

[43]  Kim L. Boyer,et al.  Quantitative Measures of Change Based on Feature Organization: Eigenvalues and Eigenvectors , 1998, Comput. Vis. Image Underst..

[44]  David J. Kriegman,et al.  What is the set of images of an object under all possible lighting conditions? , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[45]  Anil K. Jain.  Data clustering: 50 years beyond K-means , 2008, Pattern Recognit. Lett..

[46]  Vojtech Rödl,et al.  Hypergraphs do not jump , 1984, Comb..

[47]  Pietro Perona,et al.  A Factorization Approach to Grouping , 1998, ECCV.

[48]  Andrea Torsello,et al.  Matching as a non-cooperative game , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[49]  Jörgen W. Weibull,et al.  Evolutionary Game Theory , 1996 .

[50]  L. Baum,et al.  A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains , 1970 .

[51]  Radu Horaud,et al.  Figure-Ground Discrimination: A Combinatorial Optimization Approach , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[52]  Andrea Torsello,et al.  Beyond partitions: Allowing overlapping groups in pairwise clustering , 2008, 2008 19th International Conference on Pattern Recognition.