Automatic synthesis of out-of-core algorithms

We present a system for the automatic synthesis of efficient algorithms specialized for a particular memory hierarchy and a set of storage devices. The developer provides two independent inputs: 1) an algorithm that ignores memory hierarchy and external storage aspects; and 2) a description of the target memory hierarchy, including its topology and parameters. Our system is able to automatically synthesize memory-hierarchy and storage-device-aware algorithms out of those specifications, for tasks such as joins and sorting. The framework is extensible and allows developers to quickly synthesize custom out-of-core algorithms as new storage technologies become available.

[1]  Nils Anders Danielsson.  Lightweight semiformal time complexity analysis for purely functional data structures , 2008, POPL '08.

[2]  Ulf Assarsson,et al.  Fast parallel GPU-sorting using a hybrid algorithm , 2008, J. Parallel Distributed Comput..

[3]  Scott Boag,et al.  XQuery 1.0 : An XML Query Language , 2007 .

[4]  Dinesh Manocha,et al.  GPUTeraSort: high performance graphics co-processor sorting for large database management , 2006, SIGMOD Conference.

[5]  Lex Augusteijn,et al.  Sorting Morphisms , 1998, Advanced Functional Programming.

[6]  Val Tannen,et al.  Logical and Computational Aspects of Programming with Sets/Bags/Lists , 1991, ICALP.

[7]  Sriram Krishnamoorthy,et al.  Efficient synthesis of out-of-core algorithms using a nonlinear optimization solver , 2004, 18th International Parallel and Distributed Processing Symposium, 2004. Proceedings..

[8]  William J. Dally,et al.  A tuning framework for software-managed memory hierarchies , 2008, 2008 International Conference on Parallel Architectures and Compilation Techniques (PACT).

[9]  Kyuseok Shim,et al.  FAST: Flash-aware external sorting for mobile database systems , 2009, J. Syst. Softw..

[10]  Dongrui Fan,et al.  High performance comparison-based sorting algorithm on many-core GPUs , 2010, 2010 IEEE International Symposium on Parallel & Distributed Processing (IPDPS).

[11]  Pradeep Dubey,et al.  FAST: fast architecture sensitive tree search on modern CPUs and GPUs , 2010, SIGMOD Conference.

[12]  Marco Sciandrone,et al.  Sequential Penalty Derivative-Free Methods for Nonlinear Constrained Optimization , 2010, SIAM J. Optim..

[13]  Limsoon Wong,et al.  Naturally Embedded Query Languages , 1992, ICDT.

[14]  Maarten M. Fokkinga,et al.  Functional Programming with Bananas, Lenses, Envelopes and Barbed Wire , 1991, FPCA.

[15]  Richard S. Bird,et al.  Algebraic Identities for Program Calculation , 1989, Comput. J..

[16]  Raghu Ramakrishnan,et al.  Database Management Systems , 1976 .

[17]  Philippas Tsigas,et al.  A Practical Quicksort Algorithm for Graphics Processors , 2008, ESA.

[18]  Panos K. Chrysanthis,et al.  FSort: external sorting on flash-based sensor devices , 2009, DMSN '09.

[19]  Yang Liu,et al.  External Sorting on Flash Memory Via Natural Page Run Generation , 2011, Comput. J..

[20]  Elvira Albert,et al.  Closed-Form Upper Bounds in Static Cost Analysis , 2011, Journal of Automated Reasoning.

[21]  Martin Hofmann,et al.  Static determination of quantitative resource usage for higher-order programs , 2010, POPL '10.

[22]  Martin Hofmann,et al.  Static prediction of heap space usage for first-order functional programs , 2003, POPL '03.

[23]  David Padua,et al.  Encyclopedia of Parallel Computing , 2011 .