Relative factors in performance analysis of Java virtual machines

Many new Java runtime optimizations report relatively small, single-digit performance improvements. On modern virtual and actual hardware, however, the performance impact of an optimization can be influenced by a variety of factors in the underlying systems. Using a case study of a new garbage collection optimization in two different Java virtual machines, we show the relative effects of issues that must be taken into consideration when claiming an improvement. We examine the specific and overall performance changes due to our optimization and show how unintended side-effects can contribute to, and distort the final assessment. Our experience shows that VM and hardware concerns can generate variances of up to 9.5% in whole program execution time. Consideration of these confounding effects is critical to a good, objective understanding of Java performance and optimization.

[1]  Chen-Yong Cher,et al.  Software prefetching for mark-sweep garbage collection: hardware analysis and software redesign , 2004, ASPLOS XI.

[2]  Lance M. Berc,et al.  Continuous profiling: where have all the cycles gone? , 1997, TOCS.

[3]  Kathryn S. McKinley,et al.  Age-based garbage collection , 1999, OOPSLA '99.

[4]  Perry Cheng,et al.  The garbage collection advantage: improving program locality , 2004, OOPSLA.

[5]  Sandhya Dwarkadas,et al.  Characterizing and predicting program behavior and its variability , 2003, 2003 12th International Conference on Parallel Architectures and Compilation Techniques.

[6]  Laurie J. Hendren,et al.  An adaptive, region-based allocator for java , 2002, MSP/ISMM.

[7]  Matthias Hauswirth,et al.  Using Hardware Performance Monitors to Understand the Behavior of Java Applications , 2004, Virtual Machine Research and Technology Symposium.

[8]  Kathryn S. McKinley,et al.  Finding your cronies: static analysis for dynamic object colocation , 2004, OOPSLA.

[9]  Laurie J. Hendren,et al.  SableVM: A Research Framework for the Efficient Execution of Java Bytecode , 2001, Java Virtual Machine Research and Technology Symposium.

[10]  David Ungar.  Generation scavenging: a nondisruptive high performance storage reclamation algorithm , 1984 .

[11]  Chandra Krintz,et al.  Dynamic selection of application-specific garbage collectors , 2004, ISMM '04.

[12]  Hans-Juergen Boehm,et al.  Reducing garbage collector cache misses , 2000, ISMM '00.

[13]  Laurie Hendren,et al.  Dynamic metrics for java , 2003, OOPSLA 2003.

[14]  Perry Cheng,et al.  Myths and realities: the performance impact of garbage collection , 2004, SIGMETRICS '04/Performance '04.

[15]  Weng-Fai Wong,et al.  Compiler orchestrated prefetching via speculation and predication , 2004, ASPLOS XI.

[16]  Kathryn S. McKinley,et al.  Beltway: getting around garbage collection gridlock , 2002, PLDI '02.

[17]  Jingling Xue,et al.  Let's study whole-program cache behaviour analytically , 2002, Proceedings Eighth International Symposium on High Performance Computer Architecture.

[18]  Lieven Eeckhout,et al.  How java programs interact with virtual machines at the microarchitectural level , 2003, OOPSLA 2003.

[19]  Kathryn S. McKinley,et al.  Older-first garbage collection in practice: evaluation in Java Virtual Machine , 2002, MSP/ISMM.

[20]  Lars Thomas Hansen,et al.  Older-First Garbage Collection in Practice , 2000 .

[21]  Kevin Hammond,et al.  Spiking Your Caches , 1993, Functional Programming.

[22]  Bowen Alpern,et al.  Implementing jalapeño in Java , 1999, OOPSLA '99.

[23]  Dayong Gu,et al.  Code Layout as a Source of Noise in JVM Performance , 2005, Stud. Inform. Univ..

[24]  Brad Calder,et al.  The Strong correlation Between Code Signatures and Performance , 2005, IEEE International Symposium on Performance Analysis of Systems and Software, 2005. ISPASS 2005..

[25]  Matthias Hauswirth,et al.  Vertical profiling: understanding the behavior of object-priented applications , 2004, OOPSLA.

[26]  Rafael Dueire Lins,et al.  Garbage collection: algorithms for automatic dynamic memory management , 1996 .

[27]  Etienne Gagnon,et al.  A portable research framework for the execution of java bytecode , 2003 .

[28]  Thomas R. Gross,et al.  Using Platform-Specific Performance Counters for Dynamic Compilation , 2005, LCPC.

[29]  David M. Ungar,et al.  Generation Scavenging: A non-disruptive high performance storage reclamation algorithm , 1984, SDE 1.

[30]  Feng Qian,et al.  An adaptive, region-based allocator for java , 2002, ISMM '02.