Performance evaluation of the Sun Fire Link SMP clusters

The interconnection networks and the communication system software are critical in achieving high performance in clusters. Sun Fire Link interconnect is a memory-based interconnect, where the Sun MPI uses the Remote Shared Memory (RSM) model for its user-level inter-node messaging. This paper presents the overall architecture of the Sun Fire Link and its messaging layer. We provide an in-depth performance evaluation of a Sun Fire Link cluster at the RSM, MPI, and application layers. The MPI ping-pong latency and bandwidth are five microseconds and 660 MB/s, respectively. In general, the Sun Fire Link cluster performs relatively well in most cases.

[1]  Amith R. Mamidala,et al.  Evaluating InfiniBand performance with PCI Express , 2005, IEEE Micro.

[2]  Alan E. Charlesworth.  The Sun Fireplane Interconnect , 2002, IEEE Micro.

[3]  Dhabaleswar K. Panda,et al.  Performance Comparison of MPI Implementations over InfiniBand, Myrinet and Quadrics , 2003, ACM/IEEE SC 2003 Conference (SC'03).

[4]  Franck Cappello,et al.  MPI versus MPI+OpenMP on the IBM SP for the NAS Benchmarks , 2000, ACM/IEEE SC 2000 Conference (SC'00).

[5]  S. Sistare,et al.  Optimization of MPI Collectives on Clusters of Large-Scale SMP’s , 1999, ACM/IEEE SC 1999 Conference (SC'99).

[6]  Richard B. Gillett.  Memory Channel Network for PCI , 1996, IEEE Micro.

[7]  Mario Lauria,et al.  LogP performance characterization of fast messages atop Myrinet , 1998, Proceedings of the Sixth Euromicro Workshop on Parallel and Distributed Processing - PDP '98 -.

[8]  Ying Qian,et al.  Performance characteristics of openMP constructs, and application benchmarks on a large symmetric multiprocessor , 2003, ICS '03.

[9]  Ramesh Subramonian,et al.  LogP: towards a realistic model of parallel computation , 1993, PPOPP '93.

[10]  Chris J. Scheiman,et al.  LogGP: incorporating long messages into the LogP model—one step closer towards a realistic model for parallel computation , 1995, SPAA '95.

[11]  Ying Qian,et al.  An evaluation of the Myrinet/GM2 two-port networks , 2004, 29th Annual IEEE International Conference on Local Computer Networks.

[12]  Charles L. Seitz,et al.  Myrinet: A Gigabit-per-Second Local Area Network , 1995, IEEE Micro.

[13]  Greg J. Regnier,et al.  The Virtual Interface Architecture , 2002, IEEE Micro.

[14]  D. S. Henty,et al.  Performance of Hybrid Message-Passing and Shared-Memory Parallelism for Discrete Element Modeling , 2000, ACM/IEEE SC 2000 Conference (SC'00).

[15]  Steve Sistare,et al.  Optimization of MPI Collectives on Clusters of Large-Scale SMP's , 1999, SC.

[16]  Cezary Dubnicki,et al.  VMMC-2 : Efficient Support for Reliable, Connection-Oriented Communication , 1997 .

[17]  Fabrizio Petrini,et al.  Performance Evaluation of the Quadrics Interconnection Network , 2001, Proceedings 15th International Parallel and Distributed Processing Symposium. IPDPS 2001.

[18]  Dhabaleswar K. Panda,et al.  Performance characterization of a 10-Gigabit Ethernet TOE , 2005, 13th Symposium on High Performance Interconnects (HOTI'05).

[19]  Keith D. Underwood,et al.  A comparison of 4X InfiniBand and Quadrics Elan-4 technologies , 2004, 2004 IEEE International Conference on Cluster Computing (IEEE Cat. No.04EX935).

[20]  Kees Verstoep,et al.  Fast Measurement of LogP Parameters for Message Passing Platforms , 2000, IPDPS Workshops.

[21]  Jarek Nieplocha,et al.  One-sided communication on the myrinet-based SMP clusters using the GM message-passing library , 2001, Proceedings 15th International Parallel and Distributed Processing Symposium. IPDPS 2001.

[22]  Pete Wyckoff,et al.  Initial Performance Evaluation of the NetEffect 10 Gigabit iWARP Adapter , 2006, 2006 IEEE International Conference on Cluster Computing.

[23]  Ahmad Afsahi,et al.  REMOTE SHARED MEMORY OVER SUN FIRE LINK INTERCONNECT , 2003 .

[24]  S.J. Sistare,et al.  Ultra-High Performance Communication with MPI and the Sun Fire™ Link Interconnect , 2002, ACM/IEEE SC 2002 Conference (SC'02).

[25]  Jeffrey S. Vetter,et al.  Communication characteristics of large-scale scientific applications for contemporary cluster architectures , 2002, Proceedings 16th International Parallel and Distributed Processing Symposium.