Message-passing and shared-data programming models - wish vs. reality

Typically, only technical arguments like performance, cost or scalability are discussed if programming models and languages on high performance computing facilities are under consideration. In this paper, we investigate the impact of human factors such as personal preferences and perceptions, and personal experience on making technical decisions. We have queried a large HPC community of the Sharcnet project in Ontario in regards to general preferences and in regards to detailed usage of language features and programming style. The main result of our study is that - as often claimed in the past but never proven - shared-memory programming models and architectures appear to be the ideal for the majority of users, even if the main architecture of the project is a distributed-memory cluster. However, experience appears to be able to quickly overcome initial difficulties in using message passing.

[1]  Laxmikant V. Kale,et al.  Object-Based Adaptive Load Balancing for MPI Programs∗ , 2000 .

[2]  Laxmikant V. Kalé,et al.  Adaptive Load Balancing for MPI Programs , 2001, International Conference on Computational Science.

[3]  Bil Lewis,et al.  Threads Primer: A Guide to Multithreaded Programming , 1995 .

[4]  Ken Kennedy,et al.  A nationwide parallel computing environment , 1997, CACM.

[5]  William Gropp,et al.  MPI: The Complete Reference , Vol. 2 - The MPI-2 Extensions , 1998 .

[6]  Ronald H. Perrott,et al.  Parallel programming , 1988, International computer science series.

[7]  Rohit Chandra,et al.  Parallel programming in openMP , 2000 .

[8]  William Gropp.  The MPI-2 extensions , 1998 .

[9]  Leonid Oliker,et al.  A Comparison of Three Programming Models for Adaptive Applications on the Origin2000 , 2000, ACM/IEEE SC 2000 Conference (SC'00).

[10]  Paul Hudak,et al.  Memory coherence in shared virtual memory systems , 1989, TOCS.

[11]  Jaswinder Pal Singh,et al.  Scaling application performance on a cache-coherent multiprocessor , 1999, ISCA.

[12]  Willy Zwaenepoel,et al.  OpenMP on Networks of Workstations , 1998, Proceedings of the IEEE/ACM SC98 Conference.

[13]  Tao Yang,et al.  Adaptive Two-level Thread Management for Fast MPI Execution on Shared Memory Machines , 1999, ACM/IEEE SC 1999 Conference (SC'99).

[14]  Luiz Fernando Capretz,et al.  Shared-data or message passing computing models – A human factor in technical choices , 2002 .

[15]  Alan L. Cox,et al.  Improving Fine-Grained Irregular Shared-Memory Benchmarks by Data Reordering , 2000, ACM/IEEE SC 2000 Conference (SC'00).