Strategy Preserving Compilation for Parallel Functional Code

Graphics Processing Units (GPUs) and other parallel devices are widely available and have the potential for accelerating a wide class of algorithms. However, expert programming skills are required to achieving maximum performance. hese devices expose low-level hardware details through imperative programming interfaces where programmers explicity encode device-specific optimisation strategies. This inevitably results in non-performance-portable programs delivering suboptimal performance on other devices. Functional programming models have recently seen a renaissance in the systems community as they offer possible solutions for tackling the performance portability challenge. Recent work has shown how to automatically choose high-performance parallelisation strategies for a wide range of hardware architectures encoded in a functional representation. However, the translation of such functional representations to the imperative program expected by the hardware interface is typically performed ad hoc with no correctness guarantees and no guarantees to preserve the intended parallelisation strategy. In this paper, we present a formalised strategy-preserving translation from high-level functional code to low-level data race free parallel imperative code. This translation is formulated and proved correct within a language we call Data Parallel Idealised Algol (DPIA), a dialect of Reynolds' Idealised Algol. Performance results on GPUs and a multicore CPU show that the formalised translation process generates low-level code with performance on a par with code generated from ad hoc approaches.

[1]  John C. Reynolds,et al.  Definitional Interpreters for Higher-Order Programming Languages , 1972, ACM '72.

[2]  Stephen D. Brookes The essence of parallel Algol , 1996, Proceedings 11th Annual IEEE Symposium on Logic in Computer Science.

[3]  D. Prawitz Natural Deduction: A Proof-Theoretical Study , 1965 .

[4]  John C. Reynolds Using functor categories to generate intermediate code , 1995, POPL '95.

[5]  Olivier Danvy,et al.  On one-pass CPS transformations , 2002, Journal of Functional Programming.

[6]  Peter W. O'Hearn,et al.  Syntactic control of interference revisited , 1995, MFPS.

[7]  John C. Reynolds,et al.  Syntactic control of interference , 1978, POPL.

[8]  Sam Lindley,et al.  Generating performance portable code using rewrite rules: from high-level functional expressions to high-performance OpenCL code , 2015, ICFP.

[9]  Trevor L. McDonell Optimising Purely Functional GPU Programs (Thesis) , 2014 .

[10]  Uday S. Reddy,et al.  Passivity and independence , 1994, Proceedings Ninth Annual IEEE Symposium on Logic in Computer Science.

[11]  Uday S. Reddy,et al.  Global state considered unnecessary: An introduction to object-based semantics , 1996, LISP Symb. Comput..

[12]  Stephen D. Brookes The Essence of Parallel Algol , 2002, Inf. Comput..

[13]  Peter W. O'Hearn A Model for Syntactic Control of Interference , 1993, Math. Struct. Comput. Sci..

[14]  Guy E. Blelloch,et al.  NESL: A Nested Data-Parallel Language (Version 2.6) , 1993 .

[15]  Peter W. O'Hearn,et al.  BI as an assertion language for mutable data structures , 2001, POPL '01.

[16]  Sean Lee,et al.  NOVA: A Functional Language for Data Parallelism , 2014, ARRAY@PLDI.

[17]  Uday S. Reddy,et al.  Automata-Theoretic Semantics of Idealized Algol with Passive Expressions , 2013, MFPS.

[18]  Manuel M. T. Chakravarty,et al.  Accelerating Haskell array codes with multicore GPUs , 2011, DAMP '11.

[19]  John C. Reynolds,et al.  Types, Abstraction and Parametric Polymorphism , 1983, IFIP Congress.

[20]  G. Gentzen Untersuchungen über das logische Schließen. I , 1935 .

[21]  Jr. Guy L. Steele,et al.  Rabbit: A Compiler for Scheme , 1978 .

[22]  Kurt Keutzer,et al.  Copperhead: compiling an embedded data parallel language , 2011, PPoPP '11.

[23]  Andrew Barber,et al.  Dual Intuitionistic Linear Logic , 1996 .

[24]  Aaftab Munshi,et al.  The OpenCL specification , 2009, 2009 IEEE Hot Chips 21 Symposium (HCS).

[25]  Christopher Strachey,et al.  Fundamental Concepts in Programming Languages , 2000, High. Order Symb. Comput..

[26]  John C. Reynolds,et al.  The essence of ALGOL , 1997 .

[27]  Jean-Yves Girard,et al.  Linear Logic , 1987, Theor. Comput. Sci..

[28]  David F. Bacon,et al.  Compiling a high-level language for GPUs: (via language support for architectures and compilers) , 2012, PLDI.

[29]  Philip Wadler,et al.  Everything old is new again: quoted domain-specific languages , 2015, PEPM.

[30]  Peter W. O'Hearn,et al.  Parametricity and local variables , 1995, JACM.

[31]  Robert D. Tennent,et al.  Semantical Analysis of Specification Logic , 1985, Inf. Comput..

[32]  Bo Joel Svensson,et al.  A language for hierarchical data parallel design-space exploration on GPUs , 2016, J. Funct. Program..

[33]  Kunle Olukotun,et al.  Delite , 2014, ACM Trans. Embed. Comput. Syst..

[34]  Dan R. Ghica,et al.  Geometry of synthesis: a structured approach to VLSI design , 2007, POPL '07.