Fast approximation of betweenness centrality through sampling

Betweenness centrality is a fundamental measure in social network analysis, expressing the importance or influence of individual vertices in a network in terms of the fraction of shortest paths that pass through them. Exact computation in large networks is prohibitively expensive and fast approximation algorithms are required in these cases. We present two efficient randomized algorithms for betweenness estimation. The algorithms are based on random sampling of shortest paths and offer probabilistic guarantees on the quality of the approximation. The first algorithm estimates the betweenness of all vertices: all approximated values are within an additive factor ɛ from the real values, with probability at least 1-δ. The second algorithm focuses on the top-K vertices with highest betweenness and approximate their betweenness within a multiplicative factor ɛ, with probability at least $1-δ. This is the first algorithm that can compute such approximation for the top-K vertices. We use results from the VC-dimension theory to develop bounds to the sample size needed to achieve the desired approximations. By proving upper and lower bounds to the VC-dimension of a range set associated with the problem at hand, we obtain a sample size that is independent from the number of vertices in the network and only depends on a characteristic quantity that we call the vertex-diameter, that is the maximum number of vertices in a shortest path. In some cases, the sample size is completely independent from any property of the graph. The extensive experimental evaluation that we performed using real and artificial networks shows that our algorithms are significantly faster and much more scalable as the number of vertices in the network grows than previously presented algorithms with similar approximation guarantees.

[1]  W. Hoeffding Probability Inequalities for sums of Bounded Random Variables , 1963 .

[2]  Ira Sheldon Pohl,et al.  Bi-directional and heuristic search in path problems , 1969 .

[3]  Vladimir Vapnik,et al.  Chervonenkis: On the uniform convergence of relative frequencies of events to their probabilities , 1971 .

[4]  J. Anthonisse The rush in a directed graph , 1971 .

[5]  Leonard M. Freeman,et al.  A set of measures of centrality based upon betweenness , 1977 .

[6]  Piotr Indyk,et al.  Fast estimation of diameter and shortest paths (without matrix multiplication) , 1996, SODA '96.

[7]  Hermann Kaindl,et al.  Bidirectional Heuristic Search Reconsidered , 1997, J. Artif. Intell. Res..

[8]  Gerhard J. Woeginger,et al.  The VC-dimension of Set Systems Defined by Graphs , 1997, Discret. Appl. Math..

[9]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[10]  Yi Li,et al.  Improved bounds on the sample complexity of learning , 2000, SODA '00.

[11]  David Eppstein,et al.  Fast approximation of centrality , 2000, SODA '01.

[12]  U. Brandes A faster algorithm for betweenness centrality , 2001 .

[13]  Jiri Matousek,et al.  Lectures on discrete geometry , 2002, Graduate texts in mathematics.

[14]  Leon Peeters,et al.  Algorithms for Centrality Indices , 2004, Network Analysis.

[15]  M E J Newman,et al.  Finding and evaluating community structure in networks. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[16]  Eli Upfal,et al.  Probability and Computing: Randomized Algorithms and Probabilistic Analysis , 2005 .

[17]  Gábor Csárdi,et al.  The igraph software package for complex network research , 2006 .

[18]  Martin G. Everett,et al.  A Graph-theoretic perspective on centrality , 2006, Soc. Networks.

[19]  Karlis Freivalds,et al.  Fast and Simple Approximation of the Diameter and Radius of a Graph , 2006, WEA.

[20]  Ulrik Brandes,et al.  Centrality Estimation in Large Networks , 2007, Int. J. Bifurc. Chaos.

[21]  David A. Bader,et al.  Approximating Betweenness Centrality , 2007, WAW.

[22]  Ulrik Brandes,et al.  On variants of shortest-path betweenness centrality and their generic computation , 2008, Soc. Networks.

[23]  Peter Sanders,et al.  Better Approximation of Betweenness Centrality , 2008, ALENEX.

[24]  Maarten Löffler,et al.  Shape Fitting on Point Sets with Probability Distributions , 2008, ESA.

[25]  Tanya Y. Berger-Wolf,et al.  Online Sampling of High Centrality Individuals in Social Networks , 2010, PAKDD.

[26]  John Skvoretz,et al.  Node centrality in weighted networks: Generalizing degree and shortest paths , 2010, Soc. Networks.

[27]  Rami Puzis,et al.  Routing betweenness centrality , 2010, JACM.

[28]  Mark Newman,et al.  Networks: An Introduction , 2010 .

[29]  Micha Sharir,et al.  Relative (p,ε)-Approximations in Geometry , 2011, Discret. Comput. Geom..

[30]  Bruno Ribeiro,et al.  Online estimating the k central nodes of a network , 2011, 2011 IEEE Network Science Workshop.

[31]  Amos Fiat,et al.  VC-Dimension and Shortest Path Algorithms , 2011, ICALP.

[32]  Liam Roditty,et al.  Approximating the diameter of a graph , 2012, ArXiv.

[33]  Adriana Iamnitchi,et al.  Identifying high betweenness centrality nodes in large social networks , 2012, Social Network Analysis and Mining.

[34]  Ameet Talwalkar,et al.  Foundations of Machine Learning , 2012, Adaptive computation and machine learning.

[35]  Kathleen M. Carley,et al.  k-Centralities: local approximations of global measures based on shortest paths , 2012, WWW.

[36]  Nick Koudas,et al.  Sampling Online Social Networks , 2013, IEEE Transactions on Knowledge and Data Engineering.

[37]  Ümit V. Çatalyürek,et al.  Shattering and Compressing Networks for Betweenness Centrality , 2013, SDM.

[38]  Eli Upfal,et al.  Efficient Discovery of Association Rules and Frequent Itemsets through Sampling with Tight Performance Guarantees , 2011, TKDD.

[39]  Yuichi Yoshida,et al.  Almost linear-time algorithms for adaptive betweenness centrality using hypergraph sketches , 2014, KDD.

[40]  Christian Staudt,et al.  NetworKit: An Interactive Tool Suite for High-Performance Network Analysis , 2014, ArXiv.

[41]  Shai Ben-David,et al.  Understanding Machine Learning: From Theory to Algorithms , 2014 .

[42]  Graham Cormode,et al.  Sampling for big data: a tutorial , 2014, KDD.

[43]  Evgenios M. Kornaropoulos,et al.  Fast approximation of betweenness centrality through sampling , 2014, Data Mining and Knowledge Discovery.

[44]  Christian Staudt,et al.  Approximating Betweenness Centrality in Large Evolving Networks , 2014, ALENEX.

[45]  Henning Meyerhenke,et al.  Fully-Dynamic Approximation of Betweenness Centrality , 2015, ESA.

[46]  Jie Tang,et al.  Sampling Representative Users from Large Social Networks , 2015, AAAI.

[47]  Francesco Bonchi,et al.  Scalable online betweenness centrality in evolving graphs , 2016, ICDE.