High Performance Inverse Preconditioning

The derivation of parallel numerical algorithms for solving sparse linear systems on modern computer systems and software platforms has attracted the attention of many researchers over the years. In this paper we present an overview on the design issues of parallel approximate inverse matrix algorithms, based on an anti-diagonal “wave pattern” approach and a “fish-bone” computational procedure, for computing explicitly various families of exact and approximate inverses for solving sparse linear systems. Parallel preconditioned conjugate gradient-type schemes in conjunction with parallel approximate inverses are presented for the efficient solution of sparse linear systems. Applications of the proposed parallel methods by solving characteristic sparse linear systems on symmetric multiprocessor systems and distributed systems are discussed and the parallel performance of the proposed schemes is given, using MPI, OpenMP and Java multithreading.

[1]  Michele Benzi,et al.  A Sparse Approximate Inverse Preconditioner for the Conjugate Gradient Method , 1996, SIAM J. Sci. Comput..

[2]  George A. Gravvanis,et al.  On the performance of parallel approximate inverse preconditioning using Java multithreading techniques , 2007, Appl. Math. Comput..

[3]  Geoffrey C. Fox,et al.  MPIJAVA: An Object-Oriented JAVA Interface to MPI , 1999, IPPS/SPDP Workshops.

[4]  George A. Gravvanis,et al.  On the Design of Parallel Finite Element Approximate Inverses , 2007, PDPTA.

[5]  L. Kolotilina,et al.  Factorized Sparse Approximate Inverse Preconditionings I. Theory , 1993, SIAM J. Matrix Anal. Appl..

[6]  Ronald F. Boisvert,et al.  Algorithms for Special Tridiagonal Systems , 1991, SIAM J. Sci. Comput..

[7]  Thomas Huckle.  Efficient computation of sparse approximate inverses , 1998 .

[8]  M. Benzi.  Preconditioning techniques for large linear systems: a survey , 2002 .

[9]  Anne Greenbaum,et al.  Approximating the inverse of a matrix for use in iterative algorithms on vector processors , 1979, Computing.

[10]  George A. Gravvanis,et al.  Parallel Approximate Finite Element Inverses on Symmetric Multiprocessor Systems , 2008, ICCS.

[11]  Henk A. van der Vorst,et al.  Bi-CGSTAB: A Fast and Smoothly Converging Variant of Bi-CG for the Solution of Nonsymmetric Linear Systems , 1992, SIAM J. Sci. Comput..

[12]  George A. Gravvanis,et al.  Design and implementation of parallel approximate inverse classes using OpenMP , 2009 .

[13]  Michael J. Quinn,et al.  Parallel programming in C with MPI and OpenMP , 2003 .

[14]  George A. Gravvanis,et al.  An explicit sparse unsymmetric finite element solver , 1996 .

[15]  Y. Saad,et al.  Iterative solution of linear systems in the 20th century , 2000 .

[16]  George A. Gravvanis,et al.  Java Multithreading based Parallel Preconditioned Generalized Conjugate Gradient type methods , 2007, Sixth International Symposium on Parallel and Distributed Computing (ISPDC'07).

[17]  Victor Eijkhout,et al.  A Matrix Framework for Conjugate Gradient Methods and Some Variants of CG with Less Synchronization Overhead , 1993, PPSC.

[18]  Tao Wang,et al.  Performance analysis of Java concurrent programming: a case study of video mining system , 2006, Proceedings 20th IEEE International Parallel & Distributed Processing Symposium.

[19]  D. J. Evans,et al.  The Use of Pre-conditioning in Iterative Methods for Solving Linear Equations with Symmetric Positive Definite Matrices , 1968 .

[20]  George A. Gravvanis.  The rate of convergence of explicit approximate inverse preconditioning , 1996, Int. J. Comput. Math..

[21]  D. Evans.  Preconditioning Methods: Theory and Applications , 1983 .

[22]  David E. Keyes,et al.  Proceedings of the Sixth SIAM Conference on Parallel Processing for Scientific Computing, PPSC 1993, Norfolk, Virginia, USA, March 22-24, 1993 , 1993, PPSC.

[23]  Kyriacos D. Papailiou,et al.  Computational fluid dynamics '98 , 1998 .

[24]  Dirk Grunwald,et al.  Efficient barriers for distributed shared memory computers , 1994, Proceedings of 8th International Parallel Processing Symposium.

[25]  Anthony T. Chronopoulos,et al.  s-step iterative methods for symmetric linear systems , 1989 .

[26]  I. Duff,et al.  Direct Methods for Sparse Matrices , 1987 .

[27]  P. Sonneveld.  CGS, A Fast Lanczos-Type Solver for Nonsymmetric Linear systems , 1989 .

[28]  George A. Gravvanis.  Explicit isomorphic iterative methods for solving arrow-type linear systems , 2000, Int. J. Comput. Math..

[29]  K. M. Giannoutakis,et al.  Fast Parallel Finite Element Approximate Inverses , 2008 .

[30]  George A. Gravvanis,et al.  Matching high performance approximate inverse preconditioning to architectural platforms , 2007, The Journal of Supercomputing.

[31]  George A. Gravvanis,et al.  An approximate inverse matrix technique for arrowhead matrices , 1998, Int. J. Comput. Math..

[32]  H. V. D. Vorst,et al.  High Performance Preconditioning , 1989 .

[33]  George A. Gravvanis,et al.  On the performance of parallel normalized explicit preconditioned conjugate gradient type methods , 2006, Proceedings 20th IEEE International Parallel & Distributed Processing Symposium.

[34]  Jack Dongarra,et al.  Numerical Linear Algebra for High-Performance Computers , 1998 .

[35]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[36]  George K. Thiruvathukal,et al.  High Performance Java Platform Computing , 2000 .

[37]  George A. Gravvanis,et al.  Explicit approximate inverse preconditioning techniques , 2002 .

[38]  Elias A. Lipitakis,et al.  Solving linear finite element systems by normalized approximate matrix factorization semi-direct methods , 1984 .

[39]  J. Mark Bull,et al.  Benchmarking Java against C and Fortran for scientific applications , 2003, Concurr. Comput. Pract. Exp..

[40]  Yves Robert,et al.  Parallel and vector conjugate gradient-like algorithms for sparse nonsymmetric linear systems , 1988, ICS '88.

[41]  Mark Kambites,et al.  JOMP—an OpenMP-like interface for Java , 2000, JAVA '00.

[42]  D. J. Evans,et al.  Explicit semi-direct methods based on approximate inverse matrix techniques for solving boundary-value problems on parallel processors , 1987 .

[43]  Selim G. Akl.  Parallel computation: models and methods , 1997 .

[44]  Elias A. Lipitakis.  A normalized sparse linear equations solver , 1983 .

[45]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[46]  Max Planitz,et al.  Acta numerica 1993 , edited by A. Iserles. Pp. 326. £24.95 1993. ISBN 0-521-44356-3 (hardback) (Cambridge University Press) - Acta numerica 1994 , edited by A. Iserles. Pp. 572. £27.95 1994. ISBN 0-521-46181-2 (hardback) (Cambridge University Press) , 1995, The Mathematical Gazette.

[47]  James Demmel,et al.  Parallel numerical linear algebra , 1993, Acta Numerica.

[48]  George A. Gravvanis,et al.  Normalized explicit finite element approximate inverse preconditioning , 2004 .

[49]  George A. Gravvanis,et al.  Explicit preconditioned iterative methods for solving large unsymmetric finite element systems , 2005, Computing.

[50]  D. Marr,et al.  Hyper-Threading Technology Architecture and MIcroarchitecture , 2002 .

[51]  T. Huckle.  Approximate sparsity patterns for the inverse of a matrix and preconditioning , 1999 .

[52]  B. Parlett,et al.  The use of a refined error bound when updating eigenvalues of tridiagonals , 1985 .

[53]  George A. Gravvanis.  OpenMP based parallel normalized direct methods for sparse finite element linear systems , 2008, The Journal of Supercomputing.

[54]  George A. Gravvanis,et al.  An Improved Parallel Algorithm for Computing Approximate Inverses by Reducing Synchronizations , 2008, 2008 International Symposium on Parallel and Distributed Computing.

[55]  George A. Gravvanis,et al.  Normalized explicit approximate inverse preconditioning for solving 3D boundary value problems on uniprocessor and distributed systems , 2006 .

[56]  Marcus J. Grote,et al.  Parallel Preconditioning with Sparse Approximate Inverses , 1997, SIAM J. Sci. Comput..

[57]  George A. Gravvanis,et al.  Parallel Exact and Approximate Arrow-Type Inverses on Symmetric Multiprocessor Systems , 2006, International Conference on Computational Science.

[58]  G. Stewart,et al.  Computing the eigenvalues and eigenvectors of symmetric arrowhead matrices , 1990 .

[59]  George A. Gravvanis,et al.  Performability Evaluation of Multitasking and Multiprocessor Systems by Explicit Approximate Inverses , 2003, PDPTA.

[60]  A. Griewank,et al.  Approximate inverse preconditionings for sparse linear systems , 1992 .

[61]  Jack Dongarra,et al.  Performance of various computers using standard sparse linear equations solving techniques , 1993 .

[62]  George A. Gravvanis,et al.  High performance finite element approximate inverse preconditioning , 2008, Appl. Math. Comput..

[63]  George A. Gravvanis,et al.  Parallel approximate finite element inverse preconditioning on distributed systems , 2004, Third International Symposium on Parallel and Distributed Computing/Third International Workshop on Algorithms, Models and Tools for Parallel Computing on Heterogeneous Networks.

[64]  Iain S. Duff,et al.  The Impact of High Performance Computing in the Solution of Linear Systems: Trends and Problems , 2000 .

[65]  George A. Gravvanis,et al.  On the rate of convergence and complexity of finite element normalized explicit approximate inverse preconditioning , 2003 .

[66]  George A. Gravvanis,et al.  Java multithreading-based parallel approximate arrow-type inverses , 2008 .

[67]  J. Mark Bull,et al.  Benchmarking Java against C and Fortran for scientific applications , 2001, JGI '01.