Performance analysis for languages hosted on the truffle framework

It is attractive to host new or existing language implementations on top of, or reusing components of, existing managed language runtimes such as the Java Virtual Machine (JVM) or the Microsoft Common Language Infrastructure (CLI). A benefit is that software development effort may be reduced, as only one managed language runtime needs to be optimised and maintained, instead of a separate compiler/runtime for each language implementation. For example, the Truffle framework combined with a JVM offers support for executing Javascript, Ruby, R, LLVM IR-compiled languages, etc., as well as supporting the execution of applications combining multiple programming languages within a Polyglot application. In trying to understand the runtime performance of Sulong (i.e. the Truffle project which enables LLVM IR execution), we found a lack of tools and guidance. A similar situation is found for benchmarks written in Ruby and R when executed as Truffle hosted languages. Further, it is non-trivial to relate performance back to the hosted language source code, and to determine whether JVM service overheads, such as garbage collection or JIT compilation, are significant. We describe how to visually analyse the performance of Truffle-hosted languages based on Flamegraphs, allowing time to be related to sampled call-stacks. We use the Linux tool perf and the JVM agent perf-map-agent, along with enhancements to the Graal JIT compiler that map sampled call-stacks onto JVM-hosted guest language source code. This paper demonstrates the ease and flexibility of using these modified tools, with low overhead during execution time. We also illustrate applicability of the techniques to understand the performance of Polyglot applications.