A Framework for the Design of Parallel Adaptive Libraries on Hard Computational Problems

In this work, we present the Adaptive Multi-Selection Framework (called AMF). AMF is an API built for helping designers to develop optimized combinations of multiple algorithms solving the same problem in function of the physical architecture and algorithm behavior. AMF offers a simple and generic model for developing automatic combination of algorithms. In this model, the user needs to specify the set of algorithms to be combined and a representative benchmark of instances of the problem solved by the algorithms. This generic solution has advantages over many existing solutions for making automatic combination that are specific to a fixed set of algorithms or computational problems. Automatic combinations of algorithms are made in AMF with the multi-selection technique. For each instance of a computational problem, its resolution under multi-selection includes a selection of a subset of candidate algorithms followed by a concurrent run of the selected algorithms with a smart resource sharing. The resource sharing is decided according to the physical architecture, the problem instance and the time allowed to compute it. The multi-selection strategy provides excellent results when there is a large variance of execution time per instance. The actual implementation of AMF is built for shared memory architectures. However, it can be extended to distributed ones. The AMF principles have been validated in particular on the well-known Constraint Satisfaction Problem.

[1]  Jack J. Dongarra,et al.  Automated empirical optimizations of software and the ATLAS project , 2001, Parallel Comput..

[2]  Denis Trystram,et al.  Combining Numerical Iterative Solvers , 2009, PARCO.

[3]  Denis Trystram,et al.  Optimizing resource sharing on cooperative execution of algorithms , 2011 .

[4]  Victor Eijkhout,et al.  The Component Structure of a Self-Adapting Numerical Software System , 2005, International Journal of Parallel Programming.

[5]  Jean-Louis Pazat,et al.  A Framework for Dynamic Adaptation of Parallel Components , 2005, PARCO.

[6]  David A. Padua,et al.  A dynamically tuned sorting library , 2004, International Symposium on Code Generation and Optimization, 2004. CGO 2004..

[7]  Marin Bougeret,et al.  Combining multiple heuristics on discrete resources , 2009, 2009 IEEE International Symposium on Parallel & Distributed Processing.

[8]  Steven G. Johnson,et al.  FFTW: an adaptive software architecture for the FFT , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[9]  Sivan Toledo,et al.  An automatically-tuned sorting library , 2007 .

[10]  Graham Kendall,et al.  Exploring Hyper-heuristic Methodologies with Genetic Programming , 2009 .

[11]  Edmund K. Burke,et al.  Adaptive automated construction of hybrid heuristics for exam timetabling and graph colouring problems , 2009, Eur. J. Oper. Res..

[12]  Victor Eijkhout,et al.  Self-adapting numerical software (SANS) effort , 2006, IBM J. Res. Dev..

[13]  Nancy M. Amato,et al.  STAPL: An Adaptive, Generic Parallel C++ Library , 2001, LCPC.

[14]  Sanjukta Bhowmick,et al.  The Role of Multi-method Linear Solvers in PDE-based Simulations , 2003, ICCSA.

[15]  Tad Hogg,et al.  An Economics Approach to Hard Computational Problems , 1997, Science.

[16]  M. V. Wilkes,et al.  The Art of Computer Programming, Volume 3, Sorting and Searching , 1974 .

[17]  Kevin Leyton-Brown,et al.  SATzilla: Portfolio-based Algorithm Selection for SAT , 2008, J. Artif. Intell. Res..

[18]  Julien Bernard,et al.  On-Line Adaptive Parallel Prefix Computation , 2006, Euro-Par.

[19]  Timothy J. Purcell.  Sorting and searching , 2005, SIGGRAPH Courses.

[20]  Victor Eijkhout,et al.  Self-Adapting Linear Algebra Algorithms and Software , 2005, Proceedings of the IEEE.