Parallel Particle Swarm Optimization for Community Detection in Large-Scale Networks

Community detection has great applications in many areas. Many algorithms have been proposed to solve this problem, while these algorithms could not detect communities in large-scale networks effectively and efficiently. In this paper, a parallel particle swarm optimization algorithm based on Apache Spark for community detection is put forward. In this algorithm, an effective representation and a specific updating strategy of discrete particle swarm optimization for parallel computing are designed. Modularity density is used as the objective function and GraphX is used to optimize modularity density parallel. To demonstrate the effectiveness of the proposed algorithm, various experiments on small real-world networks with ground-truth communities are carried out. At the same time, we test the performance of the proposed algorithm on large-scale networks. The experimental results indicate that the proposed method is effective and suitable for community detection in large-scale networks.

[1]  Maoguo Gong,et al.  Complex Network Clustering by Multiobjective Discrete Particle Swarm Optimization Based on Decomposition , 2014, IEEE Transactions on Evolutionary Computation.

[2]  Fang Wu,et al.  Finding communities in linear time: a physics approach , 2003, ArXiv.

[3]  Clara Pizzuti,et al.  GA-Net: A Genetic Algorithm for Community Detection in Social Networks , 2008, PPSN.

[4]  Erik M Bollt,et al.  Local method for detecting communities. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[5]  S. Fortunato,et al.  Resolution limit in community detection , 2006, Proceedings of the National Academy of Sciences.

[6]  Luonan Chen,et al.  Quantitative function for community detection. , 2008 .

[7]  Konstantin Avrachenkov,et al.  Cooperative Game Theory Approaches for Network Partitioning , 2017, COCOON.

[8]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[9]  M. Newman,et al.  Finding community structure in very large networks. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[10]  Inderjit S. Dhillon,et al.  Kernel k-means: spectral clustering and normalized cuts , 2004, KDD.

[11]  M E J Newman,et al.  Community structure in social and biological networks , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[12]  W. Zachary,et al.  An Information Flow Model for Conflict and Fission in Small Groups , 1977, Journal of Anthropological Research.

[13]  Ameet Talwalkar,et al.  MLlib: Machine Learning in Apache Spark , 2015, J. Mach. Learn. Res..

[14]  Maoguo Gong,et al.  Memetic algorithm for community detection in networks. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[15]  D. Lusseau,et al.  The bottlenose dolphin community of Doubtful Sound features a large proportion of long-lasting associations , 2003, Behavioral Ecology and Sociobiology.

[16]  Maoguo Gong,et al.  Greedy discrete particle swarm optimization for large-scale social network clustering , 2015, Inf. Sci..

[17]  Cheng Soon Ong,et al.  Multivariate spearman's ρ for aggregating ranks using copulas , 2016 .

[18]  Santo Fortunato,et al.  Community detection in graphs , 2009, ArXiv.

[19]  Qingfu Zhang,et al.  Community detection in networks by using multiobjective evolutionary algorithm with decomposition , 2012 .

[20]  Clara Pizzuti,et al.  A Multiobjective Genetic Algorithm to Find Communities in Complex Networks , 2012, IEEE Transactions on Evolutionary Computation.