High-Performance, Low-Complexity Deadlock Avoidance for Arbitrary Topologies/Routings

Recently, the use of graph-based network topologies has been proposed as an alternative to traditional networks such as tori or fat-trees due to their very good topological characteristics. However they pose practical implementation challenges such as the lack of deadlock avoidance strategies. Previous proposals either lack flexibility, underutilise network resources or are exceedingly complex. We propose--and prove formally--three generic, low-complexity deadlock avoidance mechanisms that only require local information. Our methods are topology- and routing-independent and their virtual channel count is bounded by the length of the longest path. We evaluate our algorithms through an extensive simulation study to measure the impact on the performance using both synthetic and realistic traffic. First we compare against a well-known HPC mechanism for dragonfly and achieve similar performance level. Then we moved to Graph-based networks and show that our mechanisms can greatly outperform traditional, spanning-tree based mechanisms, even if these use a much larger number of virtual channels. Overall, our proposal provides a simple, flexible and high performance deadlock-avoidance solution.

[1]  Javier Navaridas,et al.  Simulating and evaluating interconnection networks with INSEE , 2011, Simul. Model. Pract. Theory.

[2]  Charles E. Hughes,et al.  Interconnections , 2011 .

[3]  Javier Navaridas,et al.  A CAM-Free Exascalable HPC Router for Low-Energy Communications , 2018, ARCS.

[4]  Hideharu Amano,et al.  A Novel Channel Assignment Method to Ensure Deadlock-Freedom for Deterministic Routing , 2017, IEICE Trans. Inf. Syst..

[5]  William J. Dally,et al.  Deadlock-Free Message Routing in Multiprocessor Interconnection Networks , 1987, IEEE Transactions on Computers.

[6]  Hideharu Amano,et al.  L-turn routing: an adaptive routing in irregular networks , 2001, International Conference on Parallel Processing, 2001..

[7]  Pramodita Sharma 2012 , 2013, Les 25 ans de l’OMC: Une rétrospective en photos.

[8]  Radia Perlman,et al.  Interconnections: Bridges, Routers, Switches, and Internetworking Protocols , 1999 .

[9]  Satoshi Matsuoka,et al.  Routing on the Dependency Graph: A New Approach to Deadlock-Free High-Performance Routing , 2016, HPDC.

[10]  Leslie G. Valiant,et al.  A Scheme for Fast Parallel Communication , 1982, SIAM J. Comput..

[11]  William J. Dally,et al.  Technology-Driven, Highly-Scalable Dragonfly Topology , 2008, 2008 International Symposium on Computer Architecture.

[12]  Xin Yuan,et al.  Random Regular Graph and Generalized De Bruijn Graph with k-Shortest Path Routing , 2018, IEEE Trans. Parallel Distributed Syst..

[13]  Antonio Robles,et al.  Improving InfiniBand Routing through Multiple Virtual Networks , 2002, ISHPC.

[14]  Cruz Izu,et al.  The Adaptive Bubble Router , 2001, J. Parallel Distributed Comput..

[15]  Olav Lysne,et al.  Load Balancing of Irregular System Area Networks through Multiple Roots , 2001 .

[16]  Xin Yuan,et al.  Random Regular Graph and Generalized De Bruijn Graph with $k$ -Shortest Path Routing , 2016, IEEE Transactions on Parallel and Distributed Systems.

[17]  Ramón Beivide,et al.  Random Folded Clos Topologies for Datacenter Networks , 2017, 2017 IEEE International Symposium on High Performance Computer Architecture (HPCA).

[18]  G. Edward Suh,et al.  Application-aware deadlock-free oblivious routing , 2009, ISCA '09.

[19]  Ankit Singla,et al.  Jellyfish: Networking Data Centers Randomly , 2011, NSDI.

[20]  Antonio Robles,et al.  LASH-TOR: a generic transition-oriented routing algorithm , 2004, Proceedings. Tenth International Conference on Parallel and Distributed Systems, 2004. ICPADS 2004..

[21]  Ludmila Cherkasova,et al.  Fibre channel fabrics: evaluation and design , 1996, Proceedings of HICSS-29: 29th Hawaii International Conference on System Sciences.

[22]  Lixin Gao,et al.  DPillar: Dual-port server interconnection network for large scale data centers , 2012, Comput. Networks.

[23]  Sven-Arne Reinemo,et al.  sFtree: A fully connected and deadlock-free switch-to-switch routing algorithm for fat-trees , 2012, TACO.

[24]  Mateo Valero,et al.  OFAR-CM: Efficient Dragonfly Networks with Simple Congestion Management , 2013, 2013 IEEE 21st Annual Symposium on High-Performance Interconnects.

[25]  Haitao Wu,et al.  BCube: a high performance, server-centric network architecture for modular data centers , 2009, SIGCOMM '09.

[26]  Olav Lysne,et al.  Layered shortest path (LASH) routing in irregular system area networks , 2002, Proceedings 16th International Parallel and Distributed Processing Symposium.

[27]  Haitao Wu,et al.  Scalable and Cost-Effective Interconnection of Data-Center Servers Using Dual Server Ports , 2011, IEEE/ACM Transactions on Networking.

[28]  Dong Xiang,et al.  Multiple spanning tree construction for deadlock-free adaptive routing in irregular networks , 2012, 2012 IEEE 10th International Symposium on Parallel and Distributed Processing with Applications.

[29]  Lei Shi,et al.  Dcell: a scalable and fault-tolerant network structure for data centers , 2008, SIGCOMM '08.

[30]  Michael Burrows,et al.  Autonet: A High-Speed, Self-Configuring Local Area Network Using Point-to-Point Links , 1991, IEEE J. Sel. Areas Commun..

[31]  Federico Silla,et al.  High-Performance Routing in Networks of Workstations with Irregular Topology , 2000, IEEE Trans. Parallel Distributed Syst..

[32]  Loren Schwiebert Deadlock-Free Oblivious Wormhole Routing with Cyclic Dependencies , 2001, IEEE Trans. Computers.

[33]  Torsten Hoefler,et al.  Deadlock-Free Oblivious Routing for Arbitrary Topologies , 2011, 2011 IEEE International Parallel & Distributed Processing Symposium.

[34]  Pier Stanislao Paolucci,et al.  The Next Generation of Exascale-Class Systems: The ExaNeSt Project , 2017, 2017 Euromicro Conference on Digital System Design (DSD).