Application of Alternating Decision Trees in Selecting Sparse Linear Solvers

The solution of sparse linear systems, a fundamental and resource-intensive task in scientific computing, can be approached through multiple algorithms. Using an algorithm well adapted to characteristics of the task can significantly enhance the performance, such as reducing the time required for the operation, without compromising the quality of the result. However, the “best” solution method can vary even across linear systems generated in course of the same PDE-based simulation, thereby making solver selection a very challenging problem. In this paper, we use a machine learning technique, Alternating Decision Trees (ADT), to select efficient solvers based on the properties of sparse linear systems and runtime-dependent features, such as the stages of simulation. We demonstrate the effectiveness of this method through empirical results over linear systems drawn from computational fluid dynamics and magnetohydrodynamics applications. The results also demonstrate that using ADT can resolve the problem of “over-fitting”, which occurs when limited amount of data is available.

[1]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[2]  Owe Axelsson,et al.  A survey of preconditioned iterative methods for linear systems of algebraic equations , 1985 .

[3]  I. Duff,et al.  Direct Methods for Sparse Matrices , 1987 .

[4]  Stephen Fahrney McCormick.  Multigrid methods, theory applications, and supercomputing , 1988 .

[5]  David E. Keyes,et al.  Towards Polyalgorithmic Linear System Solvers for Nonlinear Elliptic Problems , 1994, SIAM J. Sci. Comput..

[6]  Victor Eijkhout,et al.  Algorithmic bombardment for the iterative solution of linear systems: a poly-iterative approach , 1994 .

[7]  A. Bruaset.  A survey of preconditioned iterative methods , 1995 .

[8]  Corinna Cortes,et al.  Boosting Decision Trees , 1995, NIPS.

[9]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[10]  J. Ross Quinlan,et al.  Bagging, Boosting, and C4.5 , 1996, AAAI/IAAI, Vol. 1.

[11]  C. Kelley,et al.  Convergence Analysis of Pseudo-Transient Continuation , 1998 .

[12]  L. Breiman.  Arcing Classifiers , 1998 .

[13]  L. Breiman.  Arcing classifier (with discussion and a rejoinder by the author) , 1998 .

[14]  Yoav Freund,et al.  The Alternating Decision Tree Learning Algorithm , 1999, ICML.

[15]  B. Bennett,et al.  Local Rectangular Refinement with Application to Nonreacting and Reacting Fluid Flow Problems , 1999 .

[16]  Yoav Freund,et al.  A Short Introduction to Boosting , 1999 .

[17]  John R. Rice,et al.  Enabling Technologies for Computational Science , 2000 .

[18]  W. Park,et al.  Plasma simulation studies using multilevel physics models , 1999 .

[19]  William Gropp,et al.  Globalized Newton-Krylov-Schwarz Algorithms and Software for Parallel Implicit CFD , 2000, Int. J. High Perform. Comput. Appl..

[20]  Naren Ramakrishnan,et al.  PYTHIA-II: a knowledge/database system for managing performance data and recommending scientific software , 2000, TOMS.

[21]  The Linear System Analyzer , 2000 .

[22]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[23]  Robert D. Falgout,et al.  hypre: A Library of High Performance Preconditioners , 2002, International Conference on Computational Science.

[24]  Sanjukta Bhowmick,et al.  A Combinatorial Scheme for Developing Efficient Composite Solvers , 2002, International Conference on Computational Science.

[25]  Sanjukta Bhowmick,et al.  The Role of Multi-method Linear Solvers in PDE-based Simulations , 2003, ICCSA.

[26]  P. Raghavan,et al.  Adaptive sparse linear solvers for implicit CFD using Newton-Krylov algorithms , 2003 .

[27]  Victor Eijkhout,et al.  A Proposed Standard for Numerical Metadata , 2003 .

[28]  Victor Eijkhout,et al.  Self-Adapting Numerical Software and Automatic Tuning of Heuristics , 2003, International Conference on Computational Science.

[29]  Victor Eijkhout,et al.  Self-Adapting Numerical Software for Next Generation Applications , 2003, Int. J. High Perform. Comput. Appl..

[30]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[31]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[32]  Sanjukta Bhowmick,et al.  Faster PDE-based simulations using robust composite linear solvers , 2004, Future Gener. Comput. Syst..

[33]  R. Schapire.  The Strength of Weak Learnability , 1990, Machine Learning.

[34]  Victor Eijkhout,et al.  Self-Adapting Linear Algebra Algorithms and Software , 2005, Proceedings of the IEEE.

[35]  Ian H. Witten,et al.  Data mining - practical machine learning tools and techniques, Second Edition , 2005, The Morgan Kaufmann series in data management systems.

[36]  Erika Fuentes,et al.  Statistical and Machine Learning Techniques Applied to Algorithm Selection for Solving Sparse Linear Systems , 2007 .

[37]  Tzu-Yi Chen,et al.  Neural Networks for Predicting the Behavior of Preconditioned Iterative Solvers , 2007, International Conference on Computational Science.

[38]  Tzu-Yi Chen,et al.  On Using Reinforcement Learning to Solve Sparse Linear Systems , 2008, ICCS.

[39]  Sanjukta Bhowmick,et al.  Towards Low-Cost, High-Accuracy Classifiers for Linear Solver Selection , 2009, ICCS.

[40]  Victor Eijkhout,et al.  A Standard and Software for Numerical Metadata , 2009, TOMS.

[41]  Pang-Ning Tan,et al.  Receiver Operating Characteristic , 2009, Encyclopedia of Database Systems.

[42]  Timothy A. Davis,et al.  The university of Florida sparse matrix collection , 2011, TOMS.