Galaxyfly: A Novel Family of Flexible-Radix Low-Diameter Topologies for Large-Scales Interconnection Networks

Interconnection network plays an essential role in the architecture of large-scale high performance computing (HPC) systems. In the paper, we construct a novel family of low-diameter topologies, Galaxyfly, using techniques of algebraic graphs over finite fields. Galaxyfly is guaranteed to retain a small constant diameter while achieving a flexible tradeoff between network scale and bisection bandwidth. Galaxyfly lowers the demands for high radix of network routers and is able to utilize routers with merely moderate radix to build exascale interconnection networks. We present effective congestion-aware routing algorithms for Galaxyfly by exploring its algebraic property. We conduct extensive simulations and analysis to evaluate the performance, cost and power consumption of Galaxyfly against state-of-the-art topologies. The results show that our design achieves better performance than most existing topologies under various routing algorithms and traffic patterns, and is cost-effective to deploy for exascale HPC systems.

[1]  David Blaauw,et al.  A 4.5Tb/s 3.4Tb/s/W 64×64 switch fabric with self-updating least-recently-granted priority and quality-of-service arbitration in 45nm CMOS , 2012, 2012 IEEE International Solid-State Circuits Conference.

[2]  Haitao Wu,et al.  BCube: a high performance, server-centric network architecture for modular data centers , 2009, SIGCOMM '09.

[3]  Nan Jiang,et al.  A detailed and flexible cycle-accurate Network-on-Chip simulator , 2013, 2013 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS).

[4]  Henri Casanova,et al.  A case for random shortcut topologies for HPC interconnects , 2012, 2012 39th Annual International Symposium on Computer Architecture (ISCA).

[5]  William J. Dally,et al.  Flattened butterfly: a cost-efficient topology for high-radix networks , 2007, ISCA '07.

[6]  Ankit Singla,et al.  Jellyfish: Networking Data Centers Randomly , 2011, NSDI.

[7]  Paul R. Hafner.  Geometric realisation of the graphs of McKay-Miller-Sirán , 2004, J. Comb. Theory, Ser. B.

[8]  Jung Ho Ahn,et al.  HyperX: topology, routing, and packaging of efficient large-scale networks , 2009, Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis.

[9]  Jung Ho Ahn,et al.  The role of optics in future high radix switch design , 2011, 2011 38th Annual International Symposium on Computer Architecture (ISCA).

[10]  Torsten Hoefler,et al.  Slim Fly: A Cost Effective Low-Diameter Network Topology , 2014, SC14: International Conference for High Performance Computing, Networking, Storage and Analysis.

[11]  William J. Dally,et al.  Technology-Driven, Highly-Scalable Dragonfly Topology , 2008, 2008 International Symposium on Computer Architecture.

[12]  William J. Dally,et al.  The BlackWidow High-Radix Clos Network , 2006, 33rd International Symposium on Computer Architecture (ISCA'06).

[13]  Hong Liu,et al.  Energy proportional datacenter networks , 2010, ISCA.

[14]  Henri Casanova,et al.  Augmenting low-latency HPC network with free-space optical links , 2015, 2015 IEEE 21st International Symposium on High Performance Computer Architecture (HPCA).

[15]  Mike Higgins,et al.  Cray Cascade: A scalable HPC system based on a Dragonfly network , 2012, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis.

[16]  Brendan D. McKay,et al.  A Note on Large Graphs of Diameter Two and Given Maximum Degree, , 1998, J. Comb. Theory, Ser. B.

[17]  Himanshu Shah,et al.  FireFly , 2014, SIGCOMM.

[18]  Henri Casanova,et al.  Skywalk: A Topology for HPC Networks with Low-Delay Switches , 2014, 2014 IEEE 28th International Parallel and Distributed Processing Symposium.

[19]  Henri Casanova,et al.  Layout-conscious random topologies for HPC off-chip interconnects , 2013, 2013 IEEE 19th International Symposium on High Performance Computer Architecture (HPCA).

[20]  Keith D. Underwood,et al.  Intel® Omni-path Architecture: Enabling Scalable, High Performance Fabrics , 2015, 2015 IEEE 23rd Annual Symposium on High-Performance Interconnects.

[21]  Cyriel Minkenberg,et al.  SCOC: High-radix switches made of bufferless clos networks , 2015, 2015 IEEE 21st International Symposium on High Performance Computer Architecture (HPCA).

[22]  David Blaauw,et al.  Hi-Rise: A High-Radix Switch for 3D Integration with Single-Cycle Arbitration , 2014, 2014 47th Annual IEEE/ACM International Symposium on Microarchitecture.

[23]  Xiangke Liao,et al.  High Performance Interconnect Network for Tianhe System , 2015, Journal of Computer Science and Technology.

[24]  Lei Shi,et al.  Dcell: a scalable and fault-tolerant network structure for data centers , 2008, SIGCOMM '08.