An Empirical Analysis of Iterative Solver Performance for SPD Systems

Preconditioned iterative solvers have the potential to solve very large sparse linear systems with a fraction of the memory used by direct methods. However, the effectiveness and performance of most preconditioners is not only problem dependent, but also fairly sensitive to the choice of their tunable parameters. As a result, a typical practitioner is faced with an overwhelming number of choices of solvers, preconditioners, and their parameters. The diversity of preconditioners makes it difficult to analyze them in a unified theoretical model. Hence, a systematic evaluation of existing preconditioned iterative solvers is necessary to identify the relative advantages of various implementations. We present the results of a comprehensive experimental study of the most popular preconditioner and iterative solver combinations for symmetric positive-definite systems. We introduce a methodology for a rigorous comparative evaluation of various preconditioners, including the use of some relatively simple but powerful metrics. We describe a semi-automated system for the collection, analysis, and visualization of relative performance data. The detailed comparison of various preconditioners preconditioner implementations and a state-of-the-art direct solver gives interesting insights into their relative strengths and weaknesses. We believe that these results would be useful to researchers developing preconditioners and iterative solvers as well as practitioners looking for appropriate sparse solvers for their applications.

[1]  Yousef Saad,et al.  ARMS: an algebraic recursive multilevel solver for general sparse linear systems , 2002, Numer. Linear Algebra Appl..

[2]  M. Benzi.  Preconditioning techniques for large linear systems: a survey , 2002 .

[3]  Saltelli Andrea,et al.  Global Sensitivity Analysis: The Primer , 2008 .

[4]  Tamara G. Kolda,et al.  An overview of the Trilinos project , 2005, TOMS.

[5]  W. Bangerth,et al.  deal.II—A general-purpose object-oriented finite element library , 2007, TOMS.

[6]  Miroslav Tuma,et al.  Balanced Incomplete Factorization , 2008, SIAM J. Sci. Comput..

[7]  A. Saltelli,et al.  On the Relative Importance of Input Factors in Mathematical Models , 2002 .

[8]  Y. Saad,et al.  Experimental study of ILU preconditioners for indefinite matrices , 1997 .

[9]  Jonathan J. Hu,et al.  ML 5.0 Smoothed Aggregation Users's Guide , 2006 .

[10]  Shuangzhe Liu,et al.  Global Sensitivity Analysis: The Primer by Andrea Saltelli, Marco Ratto, Terry Andres, Francesca Campolongo, Jessica Cariboni, Debora Gatelli, Michaela Saisana, Stefano Tarantola , 2008 .

[11]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[12]  V. Kumar,et al.  Parallel Threshold-based ILU Factorization , 1997, ACM/IEEE SC 1997 Conference (SC'97).

[13]  Edmond Chow,et al.  Approximate Inverse Preconditioners via Sparse-Sparse Iterations , 1998, SIAM J. Sci. Comput..

[14]  Jorge J. Moré,et al.  Benchmarking optimization software with performance profiles , 2001, Math. Program..

[15]  Vivek Sarin,et al.  A Recommendation System for Preconditioned Iterative Solvers , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[16]  E. Cuthill,et al.  Reducing the bandwidth of sparse symmetric matrices , 1969, ACM '69.

[17]  Randolph E. Bank,et al.  PLTMG - a software package for solving elliptic partial differential equations: users' guide 8.0 , 1998, Software, environments, tools.

[18]  Olof B. Widlund,et al.  Domain Decomposition Algorithms with Small Overlap , 1992, SIAM J. Sci. Comput..

[19]  Barry Smith,et al.  PETSc (Portable, Extensible Toolkit for Scientific Computation) , 2011, Encyclopedia of Parallel Computing.

[20]  José MarÍn José Mas Rafael Bru,et al.  Balanced Incomplete Factorization , 2008 .

[21]  Y. Saad,et al.  Iterative solution of linear systems in the 20th century , 2000 .

[22]  J. H. Wilkinson,et al.  AN ESTIMATE FOR THE CONDITION NUMBER OF A MATRIX , 1979 .

[23]  Anshul Gupta,et al.  Fast and effective algorithms for graph partitioning and sparse-matrix ordering , 1997, IBM J. Res. Dev..

[24]  Murat Manguoglu,et al.  PSPIKE: A Parallel Hybrid Sparse Linear System Solver , 2009, Euro-Par.

[25]  J. Pasciak,et al.  Computer solution of large sparse positive definite systems , 1982 .

[26]  Mark T. Jones,et al.  BlockSolve95 users manual: Scalable library software for the parallel solution of sparse linear systems , 1995 .

[27]  Ray S. Tuminaro,et al.  Parallel Smoothed Aggregation Multigrid : Aggregation Strategies on Massively Parallel Machines , 2000, ACM/IEEE SC 2000 Conference (SC'00).

[28]  Timothy A. Davis,et al.  The university of Florida sparse matrix collection , 2011, TOMS.

[29]  V. E. Henson,et al.  BoomerAMG: a parallel algebraic multigrid solver and preconditioner , 2002 .

[30]  M. Benzi,et al.  A comparative study of sparse approximate inverse preconditioners , 1999 .

[31]  Edmond Chow,et al.  Parallel Implementation and Practical Use of Sparse Approximate Inverse Preconditioners with a Priori Sparsity Patterns , 2001, Int. J. High Perform. Comput. Appl..

[32]  Jonathan J. Hu,et al.  ML 3.1 smoothed aggregation user's guide. , 2004 .

[33]  Sivan Toledo,et al.  An Assessment of Incomplete-LU Preconditioners for Nonsymmetric Linear Systems , 2000, Informatica.

[34]  Anshul Gupta,et al.  Adaptive Techniques for Improving the Performance of Incomplete Factorization Preconditioning , 2010, SIAM J. Sci. Comput..

[35]  Hamilton-Jacobi Equations,et al.  Multigrid Methods for , 2011 .

[36]  Gene H. Golub,et al.  Numerical solution of saddle point problems , 2005, Acta Numerica.

[37]  Alex Pothen,et al.  A Scalable Parallel Algorithm for Incomplete Factor Preconditioning , 2000, SIAM J. Sci. Comput..

[38]  Vipin Kumar,et al.  A Parallel Algorithm for Multilevel Graph Partitioning and Sparse Matrix Ordering , 1998, J. Parallel Distributed Comput..