Cluster computing: the commodity supercomputer

The availability of high‐speed networks and increasingly powerful commodity microprocessors is making the usage of clusters, or networks, of computers an appealing vehicle for cost effective parallel computing. Clusters, built using Commodity‐Off‐The‐Shelf (COTS) hardware components as well as free, or commonly used, software, are playing a major role in redefining the concept of supercomputing. In this paper we discuss the reasons why COTS‐based clusters are becoming popular environments for running supercomputing applications. We describe the current enabling technologies and present four state‐of‐the‐art cluster‐based projects. Finally, we summarise our findings and draw a number of conclusions relating to the usefulness and likely future of cluster computing. Copyright © 1999 John Wiley & Sons, Ltd.

[1]  Ewing L. Lusk,et al.  Monitors, Messages, and Clusters: The p4 Parallel Programming System , 1994, Parallel Comput..

[2]  Timothy J. Callahan.  Network Interface Specification , 1994 .

[3]  Nicholas Carriero,et al.  Linda in context , 1989, CACM.

[4]  Gregory F. Pfister,et al.  In Search of Clusters , 1995 .

[5]  Veljko M. Milutinovic,et al.  Distributed shared memory: concepts and systems , 1997, IEEE Parallel Distributed Technol. Syst. Appl..

[6]  Ken Kennedy,et al.  Compiling Fortran D for MIMD distributed-memory machines , 1992, CACM.

[7]  Shirley Browne.  Cross-Platform Parallel Debugging and Performance Analysis Tools , 1998, PVM/MPI.

[8]  William Gropp,et al.  Chameleon parallel programming tools users manual , 1993 .

[9]  José M. Bernabéu-Aubán,et al.  Solaris MC: A Multi Computer OS , 1996, USENIX Annual Technical Conference.

[10]  Mark Baker.  MPI on NT: The Current Status and Performance of the Available Environments , 1998, PVM/MPI.

[11]  Eric Anderson,et al.  Extensible, Scalable Monitoring for Clusters of Computers , 1997, LISA.

[12]  Charles L. Seitz,et al.  Myrinet: A Gigabit-per-Second Local Area Network , 1995, IEEE Micro.

[13]  Anoop Gupta,et al.  The Stanford Dash multiprocessor , 1992, Computer.

[14]  Alan L. Cox,et al.  An integrated compile-time/run-time software distributed shared memory system , 1996, ASPLOS VII.

[15]  Guy L. Steele,et al.  The High Performance Fortran Handbook , 1993 .

[16]  Seth Copen Goldstein,et al.  Active messages: a mechanism for integrating communication and computation , 1998, ISCA '98.

[17]  Helen Custer,et al.  Inside Windows NT , 1992 .

[18]  Anthony Skjellum,et al.  A High-Performance, Portable Implementation of the MPI Message Passing Interface Standard , 1996, Parallel Comput..

[19]  Miron Livny,et al.  Profiling Workstations' Available Capacity for Remote Execution , 1987, Performance.

[20]  Partha Dasgupta,et al.  The Design and Implementation of the Clouds Distributed Operating System , 1989, Comput. Syst..

[21]  Anoop Gupta,et al.  Parallel computer architecture - a hardware / software approach , 1998 .

[22]  Paul Hudak,et al.  Memory coherence in shared virtual memory systems , 1989, TOCS.

[23]  David E. Culler,et al.  A case for NOW (networks of workstation) , 1995, PODC '95.