Performance scalability and dynamic behavior of Parsec benchmarks on many-core processors

The Parsec benchmark suite is widely used in evaluation of parallel architectures, both existing and novel, the latter through simulation. In particular, it is used for evaluation of highly parallel architectures. It is well known that parallelism bottlenecks occur both in the architecture, (e.g., shared-resource contention) and in the algorithm, (e.g., data-dependency). In this paper we study the latter, i.e., the inherent parallelism scalability and the dynamic behavior of the benchmark programs themselves, independently of the architecture. To this end, we present a new simulator that performs efficient, functionally accurate, simulation of a hypothetical ideal parallel architecture with no parallelism bottlenecks, where any measured parallelism limitation is necessarily due the benchmark itself. By applying this methodology to a continuum of simulated machines, ranging from a few processors to thousands of processors, we characterize the dynamic behavior and scalability of different benchmarks. We find that only a quarter of the Parsec benchmarks truly scale well to hundreds of processors. Moreover, somewhat surprisingly, we find the Amdahl effects are responsible for lack of scaling in only about half the non-scalable benchmarks. The rest are limited by their inability to produce sufficient work for all cores, and the others benchmarks’ scalability is limited by Amdahl effects.