Parallel Sparse Supports for Array Intrinsic Functions of Fortran 90

Fortran 90 provides a rich set of array intrinsic functions. Each of these array intrinsic functions operates on the elements of multi-dimensional array objects concurrently. They provide a rich source of parallelism and play an increasingly important role in automatic support of data parallel programming. However, there is no such support if these intrinsic functions are applied to sparse data sets. In this paper, we address this open gap by presenting an efficient library for parallel sparse computations with Fortran 90 array intrinsic operations. Our method provides both compression schemes and distribution schemes on distributed memory environments applicable to higher-dimensional sparse arrays. This way, programmers need not worry about low-level system details when developing sparse applications. Sparse programs can be expressed concisely using array expressions, and parallelized with the help of our library. Our sparse libraries are built for array intrinsics of Fortran 90, and they include an extensive set of array operations such as CSHIFT, EOSHIFT, MATMUL, MERGE, PACK, SUM, RESHAPE, SPREAD, TRANSPOSE, UNPACK, and section moves. Our work, to our best knowledge, is the first work to give sparse and parallel sparse supports for array intrinsics of Fortran 90. In addition, we provide a complete complexity analysis for our sparse implementation. The complexity of our algorithms is in proportion to the number of nonzero elements in the arrays, and that is consistent with the conventional design criteria for sparse algorithms and data structures. Our current testbed is an IBM SP2 workstation cluster. Preliminary experimental results with numerical routines, numerical applications, and data-intensive applications related to OLAP (on-line analytical processing) show that our approach is promising in speeding up sparse matrix computations on both sequential and distributed memory environments if the programs are expressed with Fortran 90 array expressions.

[1]  Jenq Kuen Lee,et al.  Parallelisation of Pressure Correction Method on Unstructured Grids , 1999 .

[2]  Marina C. Chen,et al.  Compiling Communication-Efficient Programs for Massively Parallel Machines , 1991, IEEE Trans. Parallel Distributed Syst..

[3]  Marina C. Chen,et al.  The Data Alignment Phase in Compiling Programs for Distrubuted-Memory Machines , 1991, J. Parallel Distributed Comput..

[4]  Barbara M. Chapman,et al.  New data-parallel language features for sparse matrix computations , 1995, Proceedings of 9th International Parallel Processing Symposium.

[5]  Jenq Kuen Lee,et al.  Array operation synthesis to optimize HPF programs , 1996, Proceedings of the 1996 ICPP Workshop on Challenges for Parallel Processing.

[6]  John R. Gilbert,et al.  Automatic array alignment in data-parallel programs , 1993, POPL '93.

[7]  Guy L. Steele,et al.  The High Performance Fortran Handbook , 1993 .

[8]  Michael Philippsen,et al.  Data and Process Alignment in Modula-2 , 1994, Automatic Parallelization.

[9]  Keshav Pingali,et al.  Compiling Parallel Sparse Code for User-Defined Data Structures , 1997, PPSC.

[10]  Emilio L. Zapata,et al.  Efficient resolution of sparse indirections in data-parallel compilers , 1995, ICS '95.

[11]  Jenq Kuen Lee,et al.  Towards Automatic Support of Parallel Sparse Computation in Java with Continuous Compilation , 1997, Concurr. Pract. Exp..

[12]  Jenq Kuen Lee,et al.  A Function-Composition Approach to Synthesize Fortran 90 Array Operations , 1998, J. Parallel Distributed Comput..

[13]  Jenq Kuen Lee,et al.  Towards Automatic Support of Parallel Sparse Computation in Java with Continuous Compilation , 1997, Concurrency Practice and Experience.

[14]  Richard L. Wexelblat,et al.  Proceedings of the 25th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming , 1995, PPoPP 1995.

[15]  Jenq Kuen Lee,et al.  Compiler optimizations for parallel sparse programs with array intrinsics of Fortran 90 , 1999, Proceedings of the 1999 International Conference on Parallel Processing.

[16]  Vipin Kumar,et al.  ScalParC: a new scalable and efficient parallel classification algorithm for mining large datasets , 1998, Proceedings of the First Merged International Parallel Processing Symposium and Symposium on Parallel and Distributed Processing.

[17]  Alok N. Choudhary,et al.  High performance data mining using data cubes on parallel computers , 1998, Proceedings of the First Merged International Parallel Processing Symposium and Symposium on Parallel and Distributed Processing.

[18]  M. P. Levin,et al.  Numerical Recipes In Fortran 90: The Art Of Parallel Scientific Computing , 1998, IEEE Concurrency.

[19]  Samuel D. Stearns,et al.  Signal processing algorithms using Fortran and C , 1992 .

[20]  Wai-Mee Ching,et al.  An experimental APL compiler for a distributed memory parallel machine , 1994, Proceedings of Supercomputing '94.

[21]  Michael Philippsen,et al.  Automatic alignment of array data and processes to reduce communication time on DMPPs , 1995, PPOPP '95.

[22]  Joel H. Saltz,et al.  Run-Time Techniques for Parallelizing Sparse Matrix Problems , 1995, IRREGULAR.

[23]  Jenq Kuen Lee,et al.  An array operation synthesis scheme to optimize Fortran 90 programs , 1995, PPOPP '95.

[24]  Aart J. C. Bik,et al.  Automatic Data Structure Selection and Transformation for Sparse Matrix Computations , 1996, IEEE Trans. Parallel Distributed Syst..

[25]  Guy L. Steele,et al.  Data Optimization: Allocation of Arrays to Reduce Communication on SIMD Machines , 1990, J. Parallel Distributed Comput..

[26]  Jenq Kuen Lee,et al.  Sampling and Analytical Techniques for Data Distribution of Parallel Sparse Computation , 1997, PPSC.

[27]  L. M. Ni,et al.  Issues in scalable library design for massively parallel computers , 1993, Supercomputing '93.

[28]  David A. Padua,et al.  A MATLAB to Fortran 90 translator and its effectiveness , 1996, ICS '96.

[29]  John R. Gilbert,et al.  Sparse Matrices in MATLAB: Design and Implementation , 1992, SIAM J. Matrix Anal. Appl..

[30]  Jenq Kuen Lee,et al.  Efficient support of parallel sparse computation for array intrinsic functions of Fortran 90 , 1998, ICS '98.

[31]  Alok N. Choudhary,et al.  High Performance OLAP and Data Mining on Parallel Computers , 1997, Data Mining and Knowledge Discovery.