Compiling for Heterogeneous System: A Survey and an Approach

Large applications tend to contain several models of parallelism, but only a few of these map efficiently to the single model of parallelism embodied in a homogeneous parallel system. Heterogeneous parallel systems incorporate diverse models of parallelism within a single machine or across machines. These systems are already pervasive in industrial and academic settings and offer a wealth of underutilized resources for achieving high performance. Unfortunately, heterogeneity complicates software development. We believe that compilers can and should assist in managing this complexity. We identify four goals for extending compilers to assist with managing heterogeneity: exploiting available resources, targeting changing resources, adjusting optimization to suit a target, and allowing programming models and languages to evolve. These goals do not require changes to the individual pieces of a compiler so much as a restructuring of a compiler''s software architecture to increase its flexibility. We examine the features and flexibility of six important parallelizing compilers to find existing solutions for flexibility. Where no solutions exist, we propose architectural changes to compilers.