ReNets: Statically-Optimal Demand-Aware Networks

This paper studies the design of self-adjusting datacenter networks whose physical topology dynamically adapts to the workload, in an online and demand-aware manner. We propose ReNet, a self-adjusting network which does not require any predictions about future demands and amortizes reconfigurations: it performs as good as a hypothetical static algorithm with perfect knowledge of the future demand. In particular, we show that for arbitrary sparse communication demands, ReNets achieve static optimality, a fundamental property of learning algorithms, and that route lengths in ReNets are proportional to existing lower bounds, which are known to relate to an entropy metric of the demand. ReNets provide additional desirable properties such as compact and local routing and flat addressing therefore ensuring scalability and further reducing the overhead of reconfiguration. To achieve these properties, ReNets combine multiple self-adjusting tree topologies which are optimized toward individual sources, called ego-trees in this paper.

[1]  Jean C. Walrand,et al.  Achieving 100% throughput in an input-queued switch , 1996, Proceedings of IEEE INFOCOM '96. Conference on Computer Communications.

[2]  Allan Borodin,et al.  Online computation and competitive analysis , 1998 .

[3]  Jie Gao,et al.  Competitive analysis for online scheduling in software-defined optical WAN , 2017, IEEE INFOCOM 2017 - IEEE Conference on Computer Communications.

[4]  Mohit Singh,et al.  Online and Offline Greedy Algorithms for Routing with Switching Costs , 2019, ArXiv.

[5]  Paramvir Bahl,et al.  Augmenting data center networks with multi-gigabit wireless links , 2011, SIGCOMM.

[6]  Chen Avin,et al.  Distributed Self-Adjusting Tree Networks , 2019, IEEE INFOCOM 2019 - IEEE Conference on Computer Communications.

[7]  Himanshu Shah,et al.  FireFly , 2014, SIGCOMM.

[8]  Amin Vahdat,et al.  Integrating microsecond circuit switching into the data center , 2013, SIGCOMM.

[9]  Peter Auer,et al.  The Nonstochastic Multiarmed Bandit Problem , 2002, SIAM J. Comput..

[10]  N. Linial,et al.  Expander Graphs and their Applications , 2006 .

[11]  Ankit Singla,et al.  Fat-FREE Topologies , 2016, HotNets.

[12]  Ankit Singla,et al.  OSA: An Optical Switching Architecture for Data Center Networks With Unprecedented Flexibility , 2012, IEEE/ACM Transactions on Networking.

[13]  Michael Dinitz,et al.  Scheduling for Weighted Flow and Completion Times in Reconfigurable Networks , 2020, IEEE INFOCOM 2020 - IEEE Conference on Computer Communications.

[14]  Haitao Wu,et al.  BCube: a high performance, server-centric network architecture for modular data centers , 2009, SIGCOMM '09.

[15]  Torsten Hoefler,et al.  Slim Fly: A Cost Effective Low-Diameter Network Topology , 2014, SC14: International Conference for High Performance Computing, Networking, Storage and Analysis.

[16]  Ben Y. Zhao,et al.  Mirror mirror on the ceiling: flexible wireless links for data centers , 2012, SIGCOMM '12.

[17]  Haitao Wu,et al.  MDCube: a high performance network structure for modular data center interconnection , 2009, CoNEXT '09.

[18]  Janardhan Kulkarni,et al.  Scheduling Opportunistic Links in Two-Tiered Reconfigurable Datacenters , 2020, SPAA.

[19]  Y. Freund,et al.  Adaptive game playing using multiplicative weights , 1999 .

[20]  Stefan Schmid,et al.  Toward demand-aware networking: a theory for self-adjusting networks , 2018, CCRV.

[21]  Alex C. Snoeren,et al.  Inside the Social Network's (Datacenter) Network , 2015, Comput. Commun. Rev..

[22]  Nick McKeown,et al.  Matching output queueing with a combined input output queued switch , 1999, IEEE INFOCOM '99. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No.99CH36320).

[23]  Albert G. Greenberg,et al.  Data center TCP (DCTCP) , 2010, SIGCOMM '10.

[24]  Stefan Schmid,et al.  On the Complexity of Non-Segregated Routing in Reconfigurable Data Center Architectures , 2019, CCRV.

[25]  Stefan Schmid,et al.  Demand-Aware Network Designs of Bounded Degree , 2017, DISC.

[26]  Erik D. Demaine,et al.  Dynamic Optimality - Almost , 2004, FOCS.

[27]  Jeffrey Scott Vitter,et al.  Design and analysis of dynamic Huffman codes , 1987, JACM.

[28]  Pramod Viswanath,et al.  Costly circuits, submodular schedules and approximate Carathéodory Theorems , 2016, Queueing Syst. Theory Appl..

[29]  Glenn Judd,et al.  Attaining the Promise and Avoiding the Pitfalls of TCP in the Datacenter , 2015, NSDI.

[30]  Stefan Schmid,et al.  Characterizing the algorithmic complexity of reconfigurable data center architectures , 2018, ANCS.

[31]  Chen Avin,et al.  Dynamically Optimal Self-adjusting Single-Source Tree Networks , 2020, LATIN.

[32]  Albert G. Greenberg,et al.  The nature of data center traffic: measurements & analysis , 2009, IMC '09.

[33]  Shiri Chechik,et al.  Compact Routing Schemes , 2016, Encyclopedia of Algorithms.

[34]  Robert E. Tarjan,et al.  Self-adjusting binary search trees , 1985, JACM.

[35]  Ankit Singla,et al.  High Throughput Data Center Topology Design , 2013, NSDI.

[36]  Bruce M. Maggs,et al.  A Universal Approach to Data Center Network Design , 2014, SPAA.

[37]  Chen Avin,et al.  On the Complexity of Traffic Traces and Implications , 2020, SIGMETRICS.

[38]  Ankit Singla,et al.  Jellyfish: Networking Data Centers Randomly , 2011, NSDI.

[39]  Hong Liu,et al.  Jupiter Rising: A Decade of Clos Topologies and Centralized Control in Google's Datacenter Network , 2015, Comput. Commun. Rev..

[40]  Adam Meyerson,et al.  Online oblivious routing , 2003, SPAA '03.

[41]  Amin Vahdat,et al.  Helios: a hybrid electrical/optical switch architecture for modular data centers , 2010, SIGCOMM '10.

[42]  Atul Singh,et al.  Proteus: a topology malleable data center network , 2010, Hotnets-IX.

[43]  Gal Shahaf,et al.  Beyond fat-trees without antennae, mirrors, and disco-balls , 2017, SIGCOMM.

[44]  Alex C. Snoeren,et al.  RotorNet: A Scalable, Low-complexity, Optical Datacenter Network , 2017, SIGCOMM.

[45]  Manfred K. Warmuth,et al.  The Weighted Majority Algorithm , 1994, Inf. Comput..

[46]  Albert G. Greenberg,et al.  VL2: a scalable and flexible data center network , 2009, SIGCOMM '09.

[47]  Christian Scheideler,et al.  SplayNet: Towards Locally Self-Adjusting Networks , 2016, IEEE/ACM Transactions on Networking.

[48]  Rajdeep Das,et al.  Expanding across time to deliver bandwidth efficiency and low latency , 2019, NSDI.

[49]  Thomas E. Anderson,et al.  F10: A Fault-Tolerant Engineered Network , 2013, NSDI.

[50]  Nikhil R. Devanur,et al.  ProjecToR: Agile Reconfigurable Data Center Interconnect , 2016, SIGCOMM.

[51]  Keren Bergman,et al.  Flexspander: augmenting expander networks in high-performance systems with optical bandwidth steering , 2020, IEEE/OSA Journal of Optical Communications and Networking.

[52]  Lei Shi,et al.  Dcell: a scalable and fault-tolerant network structure for data centers , 2008, SIGCOMM '08.

[53]  Stefan Schmid,et al.  Efficient Non-Segregated Routing for Reconfigurable Demand-Aware Networks , 2019, 2019 IFIP Networking Conference (IFIP Networking).

[54]  Amin Vahdat,et al.  A scalable, commodity data center network architecture , 2008, SIGCOMM '08.

[55]  He Liu,et al.  Circuit Switching Under the Radar with REACToR , 2014, NSDI.

[56]  Flyways To DeCongest Data Center Networks , 2009 .

[57]  Nick McKeown,et al.  The iSLIP scheduling algorithm for input-queued switches , 1999, TNET.

[58]  Greg N. Frederickson,et al.  Designing networks with compact routing tables , 1988, Algorithmica.

[59]  Stefan Schmid,et al.  Survey of Reconfigurable Data Center Networks: Enablers, Algorithms, Complexity , 2019, SIGA.

[60]  Hitesh Ballani,et al.  Sirius: A Flat Datacenter Network with Nanosecond Optical Switching , 2020, SIGCOMM.

[61]  Stefan Schmid,et al.  Demand-Aware Network Design with Minimal Congestion and Route Lengths , 2019, IEEE INFOCOM 2019 - IEEE Conference on Computer Communications.

[62]  SchmidStefan,et al.  On the Complexity of Non-Segregated Routing in Reconfigurable Data Center Architectures , 2019 .

[63]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[64]  Paramvir Bahl,et al.  Flyways To De-Congest Data Center Networks , 2009, HotNets.

[65]  Amos Fiat,et al.  Competitive Paging Algorithms , 1991, J. Algorithms.

[66]  Xin Wang,et al.  Neural Network Meets DCN: Traffic-driven Topology Adaptation with Deep Learning , 2018, SIGMETRICS.