A Transformation--Based Approach for the Design of Parallel/Distributed Scientific Software: the FFT

We describe a methodology for designing efficient parallel and distributed scientific software. This methodology utilizes sequences of mechanizable algebra--based optimizing transformations. In this study, we apply our methodology to the FFT, starting from a high--level algebraic algorithm description. Abstract multiprocessor plans are developed and refined to specify which computations are to be done by each processor. Templates are then created that specify the locations of computations and data on the processors, as well as data flow among processors. Templates are developed in both the MPI and OpenMP programming styles. Preliminary experiments comparing code constructed using our methodology with code from several standard scientific libraries show that our code is often competitive and sometimes performs better. Interestingly, our code handled a larger range of problem sizes on one target architecture.

[1]  Sandeep K. S. Gupta,et al.  EXTENT: a portable programming environment for designing and implementing high-performance block recursive algorithms , 1994, Proceedings of Supercomputing '94.

[2]  Steve Karmesin,et al.  Optimization of Data-Parallel Field Expressions in the POOMA Framework , 1997, ISCOPE.

[3]  David A. Padua,et al.  Searching for the Best FFT Formulas with the SPL Compiler , 2000, LCPC.

[4]  Jeremy G. Siek,et al.  The Matrix Template Library: A Generic Programming Approach to High Performance Numerical Linear Algebra , 1998, ISCOPE.

[5]  Daniel J. Rosenkrantz,et al.  Compiler design theory , 1976 .

[6]  Jeffrey Scott Vitter,et al.  Algorithms for parallel memory, II: Hierarchical multilevel memories , 1992, Algorithmica.

[7]  Harry B. Hunt,et al.  Experimental Construction of a Fine-Grained Polyalgorithm for the FFT , 1999, PDPTA.

[8]  L. Mullin.  A mathematics of arrays , 1988 .

[9]  Steven G. Johnson,et al.  FFTW: an adaptive software architecture for the FFT , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[10]  David M. Nicol,et al.  Performing Out-of Core FFTs on Parallel Disk Systems , 1998, Parallel Comput..

[11]  David R. Musser,et al.  STL tutorial and reference guide , 2001 .

[12]  Steve Karmesin,et al.  Array Design and Expression Evaluation in POOMA II , 1998, ISCOPE.

[13]  R. H. H. Wester,et al.  The POOMA operating system , 1991, PRISMA Workshop.

[14]  Todd L. Veldhuizen,et al.  Expression templates , 1996 .

[15]  David A. Padua,et al.  SPL: a language and compiler for DSP algorithms , 2001, PLDI '01.

[16]  Matteo Frigo,et al.  A fast Fourier transform compiler , 1999, SIGP.

[17]  Lenore M. Restifo Mullin,et al.  Effective data parallel computation using the Psi calculus , 1996, Concurr. Pract. Exp..

[18]  Chao Lu,et al.  Mathematics of Multidimensional Fourier Transform Algorithms , 1993 .

[19]  Ramesh C. Agarwal,et al.  A high performance parallel algorithm for 1-D FFT , 1994, Proceedings of Supercomputing '94.

[20]  Matthew H. Austern.  Generic programming and the STL - using and extending the C++ standard template library , 1999, Addison-Wesley professional computing series.

[21]  Michael Conner,et al.  Recursive fast algorithm and the role of the tensor product , 1992, IEEE Trans. Signal Process..

[22]  Guang R. Gao,et al.  Multithreaded algorithms for the fast Fourier transform , 2000, SPAA '00.

[23]  C. Loan.  Computational Frameworks for the Fast Fourier Transform , 1992 .

[24]  Sandeep K. S. Gupta,et al.  On the Synthesis of Parallel Programs from Tensor Product Formulas for Block Recursive Algorithms , 1992, LCPC.

[25]  R. Tolimieri,et al.  Algorithms for Discrete Fourier Transform and Convolution , 1989 .

[26]  Charles M. Rader,et al.  Fast transforms: Algorithms, analyses, applications , 1984 .

[27]  Lenore M. Restifo Mullin,et al.  Formal method for scheduling, routing and communication protocol , 1993, [1993] Proceedings The 2nd International Symposium on High Performance Distributed Computing.

[28]  Steven G. Johnson,et al.  The Fastest Fourier Transform in the West , 1997 .

[29]  D. Miles.  Compute intensity and the FFT , 1993, Supercomputing '93.

[30]  N. Ahmed,et al.  FAST TRANSFORMS, algorithms, analysis, applications , 1983, Proceedings of the IEEE.

[31]  Larry Carter,et al.  Faster FFTs via architecture-cognizance , 2000, Proceedings 2000 International Conference on Parallel Architectures and Compilation Techniques (Cat. No.PR00622).

[32]  Sandeep K. S. Gupta,et al.  A Framework for Generating Distributed-Memory Parallel Programs for Block Recursive Algorithms , 1986, J. Parallel Distributed Comput..

[33]  Gregory V. Wilson,et al.  Parallel Programming Using C , 1996 .

[34]  Ramesh Subramonian,et al.  LogP: towards a realistic model of parallel computation , 1993, PPOPP '93.

[35]  Mark Weiss.  Algorithms, Data Structures, and Problem Solving With C++ , 1996 .

[36]  Dragan Mirkovic,et al.  An adaptive software library for fast Fourier transforms , 2000, ICS '00.

[37]  H. T. Kung,et al.  I/O complexity: The red-blue pebble game , 1981, STOC '81.

[38]  Lenore R. Mullin.  A uniform way of reasoning about array-based computation in radar: Algebraically connecting the hardware/software boundary , 2005, Digit. Signal Process..

[39]  Sandeep K. S. Gupta,et al.  Implementing Fast Fourier Transforms on Distributed-Memory Multiprocessors Using Data Redistributions , 1994, Parallel Process. Lett..

[40]  Vipin Kumar,et al.  The Scalability of FFT on Parallel Computers , 1993, IEEE Trans. Parallel Distributed Syst..

[41]  Andrew Lumsdaine,et al.  Parallel Extensions to the Matrix Template Library , 1997, PPSC.

[42]  Sharon G. Small,et al.  Four Easy Ways to a Faster FFT , 2002, J. Math. Model. Algorithms.

[43]  John E. Savage.  Extending the Hong-Kung Model to Memory Hierarchies , 1995, COCOON.

[44]  Rohit Chandra,et al.  Parallel programming in openMP , 2000 .