Type-Directed Program Synthesis and Constraint Generation for Library Portability

Fast numerical libraries have been a cornerstone of scientific computing for decades, but this comes at a price. Programs may be tied to vendor specific software ecosystems resulting in polluted, non-portable code. As we enter an era of heterogeneous computing, there is an explosion in the number of accelerator libraries required to harness specialized hardware. We need a system that allows developers to exploit ever-changing accelerator libraries, without over-specializing their code. As we cannot know the behavior of future libraries ahead of time, this paper develops a scheme that assists developers in matching their code to new libraries, without requiring the source code for these libraries. Furthermore, it can recover equivalent code from programs that use existing libraries and automatically port them to new interfaces. It first uses program synthesis to determine the meaning of a library, then maps the synthesized description into generalized constraints which are used to search the program for replacement opportunities to present to the developer. We applied this approach to existing large applications from the scientific computing and deep learning domains. Using our approach, we show speedups ranging from 1.1x to over 10x on end to end performance when using accelerator libraries.

[1]  Sylvain Paris,et al.  Helium: lifting high-performance stencil kernels from stripped x86 binaries to halide DSL code , 2015, PLDI.

[2]  Isil Dillig,et al.  Synthesizing data structure transformations from input-output examples , 2015, PLDI.

[3]  Sumit Gulwani,et al.  Synthesis of loop-free programs , 2011, PLDI '11.

[4]  Peter-Michael Osera,et al.  Type-and-example-directed program synthesis , 2015, PLDI.

[5]  Sumit Gulwani,et al.  Automating string processing in spreadsheets using input-output examples , 2011, POPL '11.

[6]  Marco Tulio Valente,et al.  APIDiff: Detecting API breaking changes , 2018, 2018 IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER).

[7]  Alexander Aiken,et al.  Stochastic superoptimization , 2012, ASPLOS '13.

[8]  Rishabh Singh,et al.  Modular Synthesis of Sketches Using Models , 2014, VMCAI.

[9]  Tjerk P. Straatsma,et al.  NWChem: A comprehensive and scalable open-source solution for large scale molecular simulations , 2010, Comput. Phys. Commun..

[10]  Amos J. Storkey,et al.  Characterising Across-Stack Optimisations for Deep Convolutional Neural Networks , 2018, 2018 IEEE International Symposium on Workload Characterization (IISWC).

[11]  Alvin Cheung,et al.  Verified lifting of stencil computations , 2016, PLDI.

[12]  John Regehr,et al.  Souper: A Synthesizing Superoptimizer , 2017, ArXiv.

[13]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Yang Wang,et al.  The Scalability-Efficiency/Maintainability-Portability Trade-Off in Simulation Software Engineering: Examples and a Preliminary Systematic Literature Review , 2016, 2016 Fourth International Workshop on Software Engineering for High Performance Computing in Computational Science and Engineering (SE-HPCCSE).

[15]  Azadeh Farzan,et al.  Synthesis of divide and conquer parallelism for loops , 2017, PLDI.

[16]  Sanjit A. Seshia,et al.  A theory of formal synthesis via inductive learning , 2015, Acta Informatica.

[17]  Niranjan Hasabnis,et al.  Checking correctness of code generator architecture specifications , 2015, 2015 IEEE/ACM International Symposium on Code Generation and Optimization (CGO).

[18]  Michael F. P. O'Boyle,et al.  CAnDL: a domain specific language for compiler analysis , 2018, CC.

[19]  Thomas W. Reps,et al.  Synthesis of machine code from semantics , 2015, PLDI.

[20]  Marsha Chechik,et al.  Tools and Algorithms for the Construction and Analysis of Systems , 2016, Lecture Notes in Computer Science.

[21]  Alvin Cheung,et al.  Synthesizing highly expressive SQL queries from input-output examples , 2017, PLDI.

[22]  Christopher D. Rosin,et al.  Stepping Stones to Inductive Synthesis of Low-Level Looping Programs , 2018, AAAI.

[23]  Rajeev Alur,et al.  Accelerating search-based program synthesis using learned probabilistic models , 2018, PLDI.

[24]  Michael F. P. O'Boyle,et al.  Automatic Matching of Legacy Code to Heterogeneous APIs: An Idiomatic Approach , 2018, ASPLOS.

[25]  J. Henkel,et al.  CatchUp! Capturing and replaying refactorings to support API evolution , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[26]  Armando Solar-Lezama,et al.  The Sketching Approach to Program Synthesis , 2009, APLAS.

[27]  Frédo Durand,et al.  Halide: a language and compiler for optimizing parallelism, locality, and recomputation in image processing pipelines , 2013, PLDI 2013.

[28]  Armando Solar-Lezama,et al.  Synthesis of Recursive ADT Transformations from Reusable Templates , 2015, TACAS.

[29]  Sumit Gulwani,et al.  Test-driven synthesis , 2014, PLDI.

[30]  Maaz Bin Safeer Ahmad,et al.  Gradual synthesis for static parallelization of single-pass array-processing programs , 2017, PLDI.

[31]  Alexander Aiken,et al.  Stratified synthesis: automatically learning the x86-64 instruction set , 2016, PLDI.

[32]  Mik Kersten,et al.  How are Java software developers using the Elipse IDE? , 2006, IEEE Software.

[33]  Cedric Nugteren,et al.  CLBlast: A Tuned OpenCL BLAS Library , 2017, IWOCL.

[34]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Koen Claessen,et al.  QuickCheck: a lightweight tool for random testing of Haskell programs , 2000, ICFP.

[36]  Armando Solar-Lezama,et al.  Program synthesis from polymorphic refinement types , 2015, PLDI.

[37]  Ali Farhadi,et al.  XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks , 2016, ECCV.

[38]  Ralph E. Johnson,et al.  The role of refactorings in API evolution , 2005, 21st IEEE International Conference on Software Maintenance (ICSM'05).