Online Balanced Repartitioning of Dynamic Communication Patterns in Polynomial Time

This paper revisits the online balanced repartitioning problem (introduced by Avin et al. at DISC 2016) which asks for a scheduler that dynamically collocates frequently communicating nodes, in order to reduce communication costs while minimizing migrations in distributed systems. More specifically, communication requests arrive online and need to be served, either remotely across different servers at cost 1, or locally within a server at cost 0; before serving a request, the online scheduler can change the mapping of nodes to servers, i.e., migrate nodes, at cost α per node move. Avin et al. presented a deterministic O (k logk)-competitive algorithm, which is optimal up to a logarithmic factor; however, their algorithm has the drawback that it relies on expensive repartitioning operations which result in a super-polynomial runtime. Our main contribution is a different deterministic algorithm pCrep which achieves the same competitive ratio, but runs in polynomial time. Our algorithm monitors the connectivity of communication requests over time, rather than the density as in prior work; this enables the polynomial runtime.We analyze pCrep both analytically and empirically.

[1]  Lucian Popa,et al.  What we talk about when we talk about cloud network performance , 2012, CCRV.

[2]  Imane Bouhaddou,et al.  A survey of clustering algorithms for an industrial context , 2019, Procedia Computer Science.

[3]  Stefan Schmid,et al.  Online Balanced Repartitioning , 2015, DISC.

[4]  Stefan Schmid,et al.  Brief Announcement: Deterministic Lower Bound for Dynamic Balanced Graph Partitioning , 2020, PODC.

[5]  Stefan Schmid,et al.  Efficient Distributed Workload (Re-)Embedding , 2019, SIGMETRICS.

[6]  Stefan Schmid,et al.  Measuring the Complexity of Packet Traces , 2019, ArXiv.

[7]  Stefan Schmid,et al.  Tight Bounds for Online Graph Partitioning , 2020, SODA.

[8]  C. Walshaw JOSTLE : parallel multilevel graph-partitioning software – an overview , 2008 .

[9]  Leah Epstein,et al.  On Variants of File Caching , 2011, ICALP.

[10]  M. Pavan,et al.  A new graph-theoretic approach to clustering and segmentation , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[11]  Weifa Liang,et al.  Efficiently computing k-edge connected components via graph decomposition , 2013, SIGMOD '13.

[12]  Stefan Schmid,et al.  Dynamic Balanced Graph Partitioning , 2015, SIAM J. Discret. Math..

[13]  Richard M. Leahy,et al.  An Optimal Graph Theoretic Approach to Data Clustering: Theory and Its Application to Image Segmentation , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  KarypisGeorge,et al.  Multilevelk-way Partitioning Scheme for Irregular Graphs , 1998 .

[15]  Amos Fiat,et al.  Competitive Paging Algorithms , 1991, J. Algorithms.

[16]  Ron Shamir,et al.  A clustering algorithm based on graph connectivity , 2000, Inf. Process. Lett..

[17]  Vipin Kumar,et al.  A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs , 1998, SIAM J. Sci. Comput..

[18]  Stefan Schmid,et al.  Competitive Clustering of Stochastic Communication Patterns on a Ring , 2017, NETYS.

[19]  Peter Sanders,et al.  Recent Advances in Graph Partitioning , 2013, Algorithm Engineering.

[20]  Mechthild Stoer,et al.  A simple min-cut algorithm , 1997, JACM.

[21]  Chris Walshaw,et al.  Mesh Partitioning: A Multilevel Balancing and Refinement Algorithm , 2000, SIAM J. Sci. Comput..

[22]  Konstantin Andreev,et al.  Balanced Graph Partitioning , 2004, SPAA '04.