Differences Between Distributed and Parallel Systems

Distributed systems have been studied for twenty years and are now coming into wider use as fast networks and powerful workstations become more readily available. In many respects a massively parallel computer resembles a network of workstations and it is tempting to port a distributed operating system to such a machine. However, there are significant differences between these two environments and a parallel operating system is needed to get the best performance out of a massively parallel system. This report characterizes the differences between distributed systems, networks of workstations, and massively parallel systems and analyzes the impact of these differences on operating system design. In the second part of the report, we introduce Puma, an operating system specifically developed for massively parallel systems. We describe Puma portals, the basic building blocks for message passing paradigms implemented on top of Puma, and show how the differences observed in the first part of the report have influenced the design and implementation of Puma.

[1]  Per Brinch Hansen,et al.  The nucleus of a multiprogramming system , 1970, CACM.

[2]  Ramesh Subramonian,et al.  LogP: towards a realistic model of parallel computation , 1993, PPOPP '93.

[3]  Philip H. Enslow.  What is a "Distributed" Data Processing System? , 1978, Computer.

[4]  Andrew P. Black,et al.  Supporting distributed applications: experience with Eden , 1985, SOSP '85.

[5]  Andrea C. Arpaci-Dusseau,et al.  Parallel programming in Split-C , 1993, Supercomputing '93. Proceedings.

[6]  David E. Culler,et al.  A case for NOW (networks of workstation) , 1995, PODC '95.

[7]  Charles E. Leiserson,et al.  Fat-trees: Universal networks for hardware-efficient supercomputing , 1985, IEEE Transactions on Computers.

[8]  B. Duzett,et al.  An overview of the nCUBE 3 supercomputer , 1992, [Proceedings 1992] The Fourth Symposium on the Frontiers of Massively Parallel Computation.

[9]  William J. Dally,et al.  The J-machine Multicomputer: An Architectural Evaluation , 1993, Proceedings of the 20th Annual International Symposium on Computer Architecture.

[10]  Bradley C. Kuszmaul,et al.  Cilk: an efficient multithreaded runtime system , 1995, PPOPP '95.

[11]  Brian N. Bershad,et al.  Extensibility safety and performance in the SPIN operating system , 1995, SOSP.

[12]  Seth Copen Goldstein,et al.  Active Messages: A Mechanism for Integrated Communication and Computation , 1992, [1992] Proceedings the 19th Annual International Symposium on Computer Architecture.

[13]  J. Dongarra.  Performance of various computers using standard linear equations software , 1990, CARN.

[14]  Robbert van Renesse,et al.  Experiences with the Amoeba distributed operating system , 1990, CACM.

[15]  David S. Greenberg,et al.  Applications of boundary element methods on the Intel Paragon , 1994, Proceedings of Supercomputing '94.

[16]  Jack J. Dongarra,et al.  Performance of various computers using standard linear equations software in a FORTRAN environment , 1988, CARN.

[17]  Scott Pakin,et al.  High Performance Messaging on Workstations: Illinois Fast Messages (FM) for Myrinet , 1995, Proceedings of the IEEE/ACM SC95 Conference.

[18]  J. Palmer,et al.  Connection Machine model CM-5 system overview , 1992, [Proceedings 1992] The Fourth Symposium on the Frontiers of Massively Parallel Computation.

[19]  Thorsten von Eicken,et al.  U-Net: a user-level network interface for parallel and distributed computing , 1995, SOSP.

[20]  Dawson R. Engler,et al.  Exokernel: an operating system architecture for application-level resource management , 1995, SOSP.

[21]  Alan Shaw,et al.  The logical design of operating systems , 1987 .

[22]  David S. Greenberg,et al.  Out of core, out of mind: practical parallel I/O , 1993, Proceedings of Scalable Parallel Libraries Conference.

[23]  William J. Bolosky,et al.  Mach: A New Kernel Foundation for UNIX Development , 1986, USENIX Summer.

[24]  Greg Henry,et al.  Massively Parallel Distributed Computing: World's First 281 Gigaflop Supercomputer , 1995 .

[25]  Wilson C. Hsieh,et al.  The persistent relevance of IPC performance: new techniques for reducing the IPC penalty , 1993, Proceedings of IEEE 4th Workshop on Workstation Operating Systems. WWOS-III.

[26]  Henry M. Levy,et al.  Limits to low-latency communication on high-speed networks , 1993, TOCS.

[27]  Anoop Gupta,et al.  Integration of message passing and shared memory in the Stanford FLASH multiprocessor , 1994, ASPLOS VI.

[28]  Dana S. Henry,et al.  A tightly-coupled processor-network interface , 1992, ASPLOS V.

[29]  Rolf Riesen,et al.  PUMA: an operating system for massively parallel systems , 1994, 1994 Proceedings of the Twenty-Seventh Hawaii International Conference on System Sciences.

[30]  Brian N. Bershad,et al.  The increasing irrelevance of IPC Performance for Micro-kernel-Based Operating Systems , 1992, USENIX Workshop on Microkernels and Other Kernel Architectures.

[31]  Andrew S. Tanenbaum,et al.  Distributed operating systems , 2009, CSUR.

[32]  Eric A. Brewer,et al.  Remote queues: exposing message queues for optimization and atomicity , 1995, SPAA '95.

[33]  William A. Wulf,et al.  HYDRA , 1974, Commun. ACM.

[34]  Message Passing Interface Forum.  MPI: A message - passing interface standard , 1994 .