Automatic Selection of Machine Learning Models for WCET-aware Compiler Heuristic Generation !

Machine learning has shown its capabilities for an automati cg enera- tion of heuristics used by optimizing compilers. The advantages of these heuristics are that they can be easily adopted to a new environment and in some cases out- perform hand-crafted compiler optimizations. However, this approach shifts the e!ort from manual heuristic tuning to the model selection pr oblem of machine learning - i.e., selecting learning algorithms and their respective parameters - which is a tedious task in its own right. In this paper, we tackle the model selection problem in a systematic way. As our experiments show, the right choice of a learning algorithm and its parame- ters can significantly a!ect the quality of the generated heu ristics. We present ag eneric framework integrating machine learning into ac ompiler to enable an automatic search for the best learning algorithm. To find goo ds ettings for the learner parameters within the large search space, optimizations based on evolu- tionary algorithms are applied. In contrast to the majority of other approaches aiming at a reduction of the average-case execution time (ACET), our goal is the minimization of the worst-case execution time (WCET) which is a key parameter for embedded systems acting as real-time systems. A careful case study on the heuristic generation for the well-known optimization loop invariant code motion shows the challenges and benefits of our methods. 1I ntroduction Optimizing compilers transform a program written in a sourc el anguage into as eman- tically equivalent program in a target language. The generated code should exhibit a high performance. Since finding optimal solutions to compiler optimizations is provably hard, compiler writers are forced to use heuristics as approximate solutions. The devel- opment of heuristics for compiler optimizations is a tediou st ask requiring both ah igh amount of expertise and an extensive trial-and-error tuning. The reasons are twofold. First, heuristics often use simplified architecture models of complex systems, which do not su!ciently capture all relevant architectural feature s. Second, compiler optimiza- tions are typically executed within a sequence of interfering optimizations. Since the mutual interactions are hardly predictable, compiler writers develop heuristics based on conservative assumptions. Such heuristics avoid negative e"ects but also prevent the exploration of the optimization potential. ! The research leading to these results has received funding from the European Community's Artist Design Network of Excellence and from the European Community's Seventh Frame-

[1]  Henrik Theiling,et al.  Design of a WCET-Aware C Compiler , 2006, 2006 IEEE/ACM/IFIP Workshop on Embedded Systems for Real Time Multimedia.

[2]  Jean-François Deverge,et al.  WCET-Directed Dynamic Scratchpad Memory Allocation of Data , 2007, 19th Euromicro Conference on Real-Time Systems (ECRTS'07).

[3]  Katharina Morik,et al.  Distributed feature extraction in a p2p setting - a case study , 2007, Future Gener. Comput. Syst..

[4]  Ingo Mierswa,et al.  Non-Convex and Multi-Objective Optimization in Data Mining - Non-Convex and Multi-Objective Optimization for Statistical Learning and Numerical Feature Engineering , 2009 .

[5]  David B. Whalley,et al.  Timing the WCET of embedded applications , 2004, Proceedings. RTAS 2004. 10th IEEE Real-Time and Embedded Technology and Applications Symposium, 2004..

[6]  François Bodin,et al.  A Machine Learning Approach to Automatic Production of Compiler Heuristics , 2002, AIMSA.

[7]  Heiko Falk,et al.  Optimal static WCET-aware scratchpad allocation of program code , 2009, 2009 46th ACM/IEEE Design Automation Conference.

[8]  Katharina Morik,et al.  Automatic WCET Reduction by Machine Learning Based Heuristics for Function Inlining , 2013 .

[9]  Saman P. Amarasinghe,et al.  Meta optimization: improving compiler heuristics with machine learning , 2003, PLDI '03.

[10]  Priti Shankar,et al.  The Compiler Design Handbook: Optimizations and Machine Code Generation , 2002, The Compiler Design Handbook.

[11]  Michael F. P. O'Boyle,et al.  Rapidly Selecting Good Compiler Optimizations using Performance Counters , 2007, International Symposium on Code Generation and Optimization (CGO'07).

[12]  Michael F. P. O'Boyle,et al.  Using machine learning to focus iterative optimization , 2006, International Symposium on Code Generation and Optimization (CGO'06).

[13]  Ting Chen,et al.  WCET centric data allocation to scratchpad memory , 2005, 26th IEEE International Real-Time Systems Symposium (RTSS'05).

[14]  Michael F. P. O'Boyle,et al.  Automatic Feature Generation for Machine Learning Based Optimizing Compilation , 2009, 2009 International Symposium on Code Generation and Optimization.

[15]  Priti Shankar,et al.  The Compiler Design Handbook: Optimizations and Machine Code Generation, Second Edition , 2007, The Compiler Design Handbook, 2nd ed..

[16]  Steven S. Muchnick,et al.  Advanced Compiler Design and Implementation , 1997 .

[17]  Michael F. P. O'Boyle,et al.  Raced profiles: efficient selection of competing compiler optimizations , 2009, LCTES '09.

[18]  L. Almagor,et al.  Finding effective compilation sequences , 2004, LCTES '04.

[19]  John Cavazos,et al.  Inducing heuristics to decide whether to schedule , 2004, PLDI '04.

[20]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[21]  José V. Busquets-Mataix,et al.  Cache contents selection for statically-locked instruction caches: an algorithm comparison , 2005, 17th Euromicro Conference on Real-Time Systems (ECRTS'05).

[22]  Björn Lisper,et al.  Data cache locking for higher program predictability , 2003, SIGMETRICS '03.

[23]  Mark Stephenson,et al.  Predicting unroll factors using supervised classification , 2005, International Symposium on Code Generation and Optimization.

[24]  Ingo Mierswa,et al.  YALE: rapid prototyping for complex data mining tasks , 2006, KDD '06.

[25]  Douglas L. Jones,et al.  Fast and efficient searches for effective optimization-phase sequences , 2005, TACO.

[26]  Y. N. Srikant,et al.  Microarchitecture Sensitive Empirical Models for Compiler Optimizations , 2007, International Symposium on Code Generation and Optimization (CGO'07).