Exploiting parallelism in functional languages: a “paradigm-oriented” approach

Deriving parallelism automatically from functional programs is simple in theory but very few practical implementations have been realised. Programs may contain too little or too much parallelism causing a degradation in performance. Such parallelism could be more eeciently controlled if parallel algorithmic structures (or skeletons) are used in the design of algorithms. A structure captures the behaviour of a parallel programming paradigm and acts as a template in the design of an algorithm. This paper presents some important parallel programming paradigms and deenes a structure for each of these paradigms. The iterative transformation paradigm (or geometric parallelism) is discussed in detail and a framework under which programs can be developed and transformed into eecient and portable implementations is presented. 1.1 The \Paradigm-Oriented" Approach In recent years, there has been a steady improvement in the design of high performance parallel computers. However, writing parallel programs is still a complex and expensive task which requires a detailed knowledge of the underlying architecture. It is now argued that functional languages could play an important role in the development of parallel applications. Their implicit parallelism eliminates the need to explicitly decompose a program into concurrent tasks, and to provide the necessary communication and synchronisation between these tasks. Following this principle, a typical parallelising compiler for functional languages would have the structure displayed in Fig. 1.1. Given an arbitrary functional program, the rst phase analyses this program to detect parallelism, and encodes decisions such as evaluation order, partitioning of data, load balancing and granularity control in the form of annotations (e.g. (Burton 1987)). There are two forms of inherent parallelism: horizontal and vertical parallelism (Kelly 1989). Horizontal parallelism evaluates