Systematic Comparison of Six Open-source Java Call Graph Construction Tools

Call graphs provide the groundwork for numerous analysis algorithms and tools. However, in practice, their construction may have several ambiguities, especially for object-oriented programming languages like Java. The characteristics of the call graphs – which are influenced by building requirements such as scalability, efficiency, completeness, and precision – can greatly affect the output of the algorithms utilizing them. Therefore, it is important for developers to know a well-defined set of criteria based on which they can choose the most appropriate call graph builder tool for their static analysis applications. In this paper, we studied and compared six static call graph creator tools for Java. Our aim was to identify linguistic and technical properties that might induce differences in the generated call graphs besides the obvious differences caused by the various call graph construction algorithms. We evaluated the tools on multiple real-life open-source Java systems and performed a quantitative and qualitative assessment of the resulting graphs. We have shown how different outputs could be generated by the different tools. By manually analyzing the differences found on larger programs, we also found differences that we did not expect based on our preliminary assumptions.