Deep Learning for Compilers First Year Review Document

Continued advancements in machine learning have increasingly extended the state-of-the art in language modelling for natural language processing. Coupled with the increasing popularity of websites such as GitHub for hosting software projects, this raises the potential for large scale language modelling over open source code to build probabilistic models which capture both the semantics of a programming language and its common usage in real world applications. This document describes my work towards the development of systems for automatically generating programs in a given programming language. The aim of this work is improvements to predictive modelling for compiler optimisation and compiler testing. In my first year I have applied LSTMs to large corpuses of code mined from open source in order to generate executable OpenCL kernels. These generated programs have been shown to improve the performance of state-of-the-art predictive models; though the programs are typically short and limited to operating only on scalars and vectors of numerical values. This document describes the plans for future work to extend this initial proof-of-concept through the use of formal grammars to generate programs in arbitrary languages, capable of satisfying arbitrary properties of interest. This will enable the automatic generation of programs in any language for a which a large corpus of real world codes is available, with a range of applications including exploration of unknown parts of program feature spaces, and identifying bugs in compilers.