Social Topic Models for Community Extraction

With social interaction playing an increasingly important role in the online world, the capability to extract latent communities based on such interactions is becoming vital for a wide variety of applications. However, existing literature on community extraction has largely focused on methods based on the link structure of a given social network. Such link-based methods ignore the content of social interactions, which may be crucial for accurate and meaningful community extraction. In this paper, we present a Bayesian generative model for community extraction which naturally incorporates both the link and content information present in the social network. The model assumes that actors in a community communicate on topics of mutual interest, and the topics of communication, in turn, determine the communities. Further, the model naturally allows actors to belong to multiple communities. The model is instantiated in the context of an email network, and a Gibbs sampling algorithm is presented to do inference. Through extensive experiments and visualization on the Enron email corpus, we demonstrate that the model is able to extract well-connected and topically meaningful communities. Additionally, the model extracts relevant topics that can be mapped back to corresponding real-life events involving Enron.

[1]  T. Snijders,et al.  Estimation and Prediction for Stochastic Blockstructures , 2001 .

[2]  M E J Newman,et al.  Community structure in social and biological networks , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[3]  Thomas L. Griffiths,et al.  Hierarchical Topic Models and the Nested Chinese Restaurant Process , 2003, NIPS.

[4]  Bernardo A. Huberman,et al.  Email as spectroscopy: automated discovery of community structure within organizations , 2003 .

[5]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[6]  M. Newman,et al.  Finding community structure in very large networks. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[7]  Mark Steyvers,et al.  Finding scientific topics , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[8]  Bernardo A. Huberman,et al.  E-Mail as Spectroscopy: Automated Discovery of Community Structure within Organizations , 2005, Inf. Soc..

[9]  Kathleen M. Carley,et al.  Exploration of communication networks from the Enron email corpus , 2005 .

[10]  Andrew McCallum,et al.  Topic and Role Discovery in Social Networks , 2005, IJCAI.

[11]  Andrew McCallum,et al.  Group and Topic Discovery from Relations and Their Attributes , 2005, NIPS.

[12]  T. Griffiths,et al.  Modeling individual differences using Dirichlet processes , 2006 .

[13]  J. Reichardt,et al.  Statistical mechanics of community detection. , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[14]  Hongyuan Zha,et al.  Probabilistic models for discovering e-communities , 2006, WWW '06.

[15]  Stanley Wasserman,et al.  Social Network Analysis: Methods and Applications , 1994, Structural analysis in the social sciences.

[16]  John D. Lafferty,et al.  A correlated topic model of Science , 2007, 0708.3601.