Extending the capabilities of zpl to improve the performance of pipelined applications

ZPL is a data-parallel array programming language created at University of Washington aimed to efficiently solve scientific computation problems on parallel computers. The portability of ZPL allows predictable performance across different parallel architectures and its high level array programming style reduces development time. Although the language is well suited for scientific and engineering problem solving, there are applications where the current ZPL parallel model would not exploit all the parallelism available. Pipelined applications often found in high-performance embedded systems, lend themselves to array processing and thus they favor the data-parallel approach of ZPL. However, such applications are stream oriented and typically involve a multi-stage pipeline of tasks that process the data. The input is a sequence of data sets that must be processed according to strict real-time constraints. The current design and implementation of ZPL does not permit pipelined applications to occur in parallel. In this doctoral dissertation we extend the capabilities of ZPL by the design and implementation of the Stream Library. The Stream Library incorporates pipelined task parallelism into the language and enables both data-parallel and pipelined-parallel models to coexist. Through the Stream Library it is now possible for ZPL to perform true pipelined computations consisting of a single chain of tasks. We ported the original ZPL environment to an embedded multiprocessor architecture and developed a native implementation of the library. To evaluate the library, we pipelined and tested four stream-processing programs. We discuss the coding strategy for pipelining these programs and present performance numbers. We show that for some conditions, the mixed model performs better than the original data-parallel model alone.