Self-falsifiable hierarchical detection of overlapping communities on social networks

No community detection algorithm can be optimal for all possible networks, thus it is important to identify whether the algorithm is suitable for a given network. We propose a multi-step algorithmic solution scheme for overlapping community detection based on an advanced label propagation process, which imitates the community formation process on social networks. Our algorithm is parameter-free and is able to reveal the hierarchical order of communities in the graph. The unique property of our solution scheme is self-falsifiability; an automatic quality check of the results is conducted after the detection, and the fitness of the algorithm for the specific network is reported. Extensive experiments show that our algorithm is self-consistent, reliable on networks of a wide range of size and different sorts, and is more robust than existing algorithms on both sparse and large-scale social networks. Results further suggest that our solution scheme may uncover features of networks' intrinsic community structures.

[1]  E. Weinan,et al.  THE LANDSCAPE OF COMPLEX NETWORKS – CRITICAL NODES AND A HIERARCHICAL DECOMPOSITION , 2014 .

[2]  Alessandro Laio,et al.  Clustering by fast search and find of density peaks , 2014, Science.

[3]  Dino Pedreschi,et al.  DEMON: a local-first discovery method for overlapping communities , 2012, KDD.

[4]  Tiago P. Peixoto Hierarchical block structures and high-resolution model selection in large networks , 2013, ArXiv.

[5]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[6]  M. Newman,et al.  Hierarchical structure and the prediction of missing links in networks , 2008, Nature.

[7]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[8]  Jure Leskovec,et al.  Community Structure in Large Networks: Natural Cluster Sizes and the Absence of Large Well-Defined Clusters , 2008, Internet Math..

[9]  Kun He,et al.  Hidden Community Detection in Social Networks , 2017, Inf. Sci..

[10]  Xiaohui Cheng,et al.  A community discovery algorithm based on boundary nodes and label propagation , 2017, Pattern Recognit. Lett..

[11]  Santo Fortunato,et al.  Network structure, metadata and the prediction of missing nodes , 2016, ArXiv.

[12]  Sune Lehmann,et al.  Link communities reveal multiscale complexity in networks , 2009, Nature.

[13]  M E J Newman,et al.  Community structure in social and biological networks , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[14]  W. Zachary,et al.  An Information Flow Model for Conflict and Fission in Small Groups , 1977, Journal of Anthropological Research.

[15]  Ji-Rong Wen,et al.  Scalable community discovery on textual data with relations , 2008, CIKM '08.

[16]  Santo Fortunato,et al.  Community detection in networks: A user guide , 2016, ArXiv.

[17]  Kun He,et al.  Local Spectral Clustering for Overlapping Community Detection , 2018, ACM Trans. Knowl. Discov. Data.

[18]  Steve Gregory,et al.  Finding overlapping communities in networks by label propagation , 2009, ArXiv.

[19]  P. Erdos,et al.  On the evolution of random graphs , 1984 .

[20]  Hans-Peter Kriegel,et al.  DBSCAN Revisited, Revisited , 2017, ACM Trans. Database Syst..

[21]  Charles A. Sutton,et al.  GEMSEC: Graph Embedding with Self Clustering , 2018, 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM).

[22]  Fan Chung Graham,et al.  Local Graph Partitioning using PageRank Vectors , 2006, 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06).

[23]  D. Lusseau,et al.  The bottlenose dolphin community of Doubtful Sound features a large proportion of long-lasting associations , 2003, Behavioral Ecology and Sociobiology.

[24]  Mark E. J. Newman,et al.  An efficient and principled method for detecting communities in networks , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[25]  Mark C. Parsons,et al.  Communicability across evolving networks. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[26]  Inderjit S. Dhillon,et al.  Overlapping community detection using seed set expansion , 2013, CIKM.

[27]  Xiaoming Liu,et al.  SLPA: Uncovering Overlapping Communities in Social Networks via a Speaker-Listener Interaction Dynamic Process , 2011, 2011 IEEE 11th International Conference on Data Mining Workshops.

[28]  James P. Bagrow Evaluating local community methods in networks , 2007, 0706.3880.

[29]  Xingyuan Wang,et al.  Detecting overlapping communities by seed community in weighted complex networks , 2013 .

[30]  B. Bollobás The evolution of random graphs , 1984 .

[31]  Ingo Scholtes,et al.  Understanding Complex Systems: From Networks to Optimal Higher-Order Models , 2018, ArXiv.

[32]  F. Radicchi,et al.  Benchmark graphs for testing community detection algorithms. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[33]  Mark E. J. Newman,et al.  Stochastic blockmodels and community structure in networks , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[34]  Réka Albert,et al.  Near linear time algorithm to detect community structures in large-scale networks. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[35]  Andrea Lancichinetti,et al.  Detecting the overlapping and hierarchical community structure in complex networks , 2008, 0802.1218.

[36]  Sean Hughes,et al.  Clustering by Fast Search and Find of Density Peaks , 2016 .

[37]  R. Lambiotte,et al.  Line graphs, link partitions, and overlapping communities. , 2009, Physical review. E, Statistical, nonlinear, and soft matter physics.

[38]  Jure Leskovec,et al.  Defining and evaluating network communities based on ground-truth , 2012, Knowledge and Information Systems.

[39]  Mao-Bin Hu,et al.  Detect overlapping and hierarchical community structure in networks , 2008, ArXiv.

[40]  Jure Leskovec,et al.  Friendship and mobility: user movement in location-based social networks , 2011, KDD.

[41]  Konstantin Avrachenkov,et al.  Cooperative Game Theory Approaches for Network Partitioning , 2017, COCOON.

[42]  Shihua Zhang,et al.  Uncovering fuzzy community structure in complex networks. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[43]  Xueqi Cheng,et al.  Exploring the structural regularities in networks , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[44]  Elchanan Mossel,et al.  Spectral redemption in clustering sparse networks , 2013, Proceedings of the National Academy of Sciences.

[45]  Leto Peel,et al.  The ground truth about metadata and community detection in networks , 2016, Science Advances.

[46]  Jon M. Kleinberg,et al.  Community membership identification from small seed sets , 2014, KDD.

[47]  Yiannis Kompatsiaris,et al.  Community detection in complex networks based on DBSCAN* and a Martingale process , 2016, 2016 11th International Workshop on Semantic and Social Media Adaptation and Personalization (SMAP).

[48]  Roger Guimerà,et al.  Extracting the hierarchical organization of complex systems , 2007, Proceedings of the National Academy of Sciences.

[49]  Ricardo J. G. B. Campello,et al.  Density-Based Clustering Based on Hierarchical Density Estimates , 2013, PAKDD.

[50]  Kuru Ratnavelu,et al.  A semi-synchronous label propagation algorithm with constraints for community detection in complex networks , 2017, Scientific Reports.

[51]  Martin Rosvall,et al.  Maps of random walks on complex networks reveal community structure , 2007, Proceedings of the National Academy of Sciences.

[52]  Santo Fortunato,et al.  Finding Statistically Significant Communities in Networks , 2010, PloS one.

[53]  Yizhou Sun,et al.  SHRINK: a structural clustering algorithm for detecting hierarchical communities in networks , 2010, CIKM.

[54]  Peilin Yang,et al.  An overlapping community detection algorithm based on density peaks , 2017, Neurocomputing.

[55]  Cristopher Moore,et al.  Community detection in networks with unequal groups , 2015, Physical review. E.

[56]  T. Vicsek,et al.  Uncovering the overlapping community structure of complex networks in nature and society , 2005, Nature.

[57]  Wenjiang Pei,et al.  Recursive filtration method for detecting community structure in networks , 2008 .

[58]  Andrea Lancichinetti,et al.  Benchmarks for testing community detection algorithms on directed and weighted graphs with overlapping communities. , 2009, Physical review. E, Statistical, nonlinear, and soft matter physics.

[59]  Fergal Reid,et al.  Percolation Computation in Complex Networks , 2012, 2012 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining.

[60]  Boleslaw K. Szymanski,et al.  Overlapping community detection in networks: The state-of-the-art and comparative study , 2011, CSUR.

[61]  A. Clauset Finding local community structure in networks. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[62]  Jean-Cédric Chappelier,et al.  Finding instabilities in the community structure of complex networks. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[63]  T. S. Evans,et al.  Clique graphs and overlapping communities , 2010, ArXiv.

[64]  Jure Leskovec,et al.  Learning to Discover Social Circles in Ego Networks , 2012, NIPS.

[65]  Christos Faloutsos,et al.  Graph evolution: Densification and shrinking diameters , 2006, TKDD.

[66]  Cristopher Moore,et al.  Scalable detection of statistically significant communities and hierarchies, using message passing for modularity , 2014, Proceedings of the National Academy of Sciences.

[67]  Qingcai Chen,et al.  Overlapping community detection in weighted networks via a Bayesian approach , 2017 .