The Development of Parkbench and Performance Prediction

The authors present a brief overview of the development of benchmarks for parallel performance analysis and show how a set of widely accepted parallel benchmarks has emerged from the Parkbench initiative. At the lowest level, basic node parameters are captured by the LINPACK benchmark and the Genesis communications benchmarks, which give information on message-passing latency and bandwidth. At kernel and application levels, results are widely available for the NAS parallel benchmarks. A new release of the Genesis communications benchmarks is described, which addresses the important issue of the effect of memory hierarchy on message transfers. The rest of the paper is concerned with the way in which benchmark results can be used to predict performance of full applications. Three studies are considered: the PERFORM estimation tool and WHITEBOX benchmarking are research projects, indicating some possible directions for progress, while INTREPID is the basis of a commercial system for scheduling meta-applications based on performance models for the components.