Optimizing the Performance-Related Configurations of Object-Relational Mapping Frameworks Using a Multi-Objective Genetic Algorithm

Object-relational mapping (ORM) frameworks map low-level database operations onto a high-level programming API that can be accessed from within object-oriented source code. ORM frameworks often provide configuration options to optimize the performance of such database operations. However, determining the set of optimal configuration options is a challenging task. Through an exploratory study on two open source applications (Spring PetClinic and ZK), we find that the difference in execution time between two configurations can be large. In addition, both applications are not shipped with an ORM configuration that is related to performance: instead, they use the default values provided by the ORM framework. We show that in 89% of the 9 analyzed test cases for PetClinic and in 96% of the 54 analyzed test cases for ZK, the default configuration values supplied by the ORM framework performed significantly slower than the optimal configuration for that test case. Based on these observations, this paper proposes an approach for automatically finding an optimal ORM configuration using a multi-objective genetic algorithm. We evaluate our approach by conducting a case study of Spring PetClinic and ZK. We find that our approach finds near-optimal configurations in 360-450 seconds for PetClinic and in 9-12 hours for ZK. These execution times allow our approach to be executed to find an optimal configuration before each new release of an application.

[1]  Yuanyuan Zhou,et al.  Do not blame users for misconfigurations , 2013, SOSP.

[2]  Bowei Xi,et al.  A smart hill-climbing algorithm for application server configuration , 2004, WWW '04.

[3]  Kewen Wang,et al.  Predator — An experience guided configuration optimizer for Hadoop MapReduce , 2012, 4th IEEE International Conference on Cloud Computing Technology and Science Proceedings.

[4]  Sven Apel,et al.  Variability-aware performance prediction: A statistical learning approach , 2013, 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[5]  Jesús García,et al.  An approach to stopping criteria for multi-objective optimization evolutionary algorithms: The MGBM criterion , 2009, 2009 IEEE Congress on Evolutionary Computation.

[6]  Vibhu Saujanya Sharma,et al.  Performance Antipatterns: Detection and Evaluation of Their Effects in the Cloud , 2014, 2014 IEEE International Conference on Services Computing.

[7]  Michael D. Ernst,et al.  Automated diagnosis of software configuration errors , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[8]  Yixin Diao,et al.  Managing Web server performance with AutoTune agents , 2003 .

[9]  Heike Trautmann,et al.  A Taxonomy of Online Stopping Criteria for Multi-Objective Evolutionary Algorithms , 2011, EMO.

[10]  Shivnath Babu,et al.  Finding Good Configurations in High-Dimensional Spaces: Doing More with Less , 2008, 2008 IEEE International Symposium on Modeling, Analysis and Simulation of Computers and Telecommunication Systems.

[11]  Wei Zheng,et al.  MassConf: automatic configuration tuning by leveraging user community information , 2011, ICPE '11.

[12]  Yi Zhang,et al.  Performance Prediction of Configurable Software Systems by Fourier Learning (T) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[13]  John Tyssedal.  Plackett‐Burman Designs , 2008 .

[14]  Gilbert Hamann,et al.  Automatic identification of load testing problems , 2008, 2008 IEEE International Conference on Software Maintenance.

[15]  Gail M. Sullivan,et al.  Using Effect Size-or Why the P Value Is Not Enough. , 2012, Journal of graduate medical education.

[16]  Kalyanmoy Deb,et al.  Multi-objective optimization using evolutionary algorithms , 2001, Wiley-Interscience series in systems and optimization.

[17]  Sven Apel,et al.  Performance-influence models for highly configurable systems , 2015, ESEC/SIGSOFT FSE.

[18]  Gunter Saake,et al.  Predicting performance via automated feature-interaction detection , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[19]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[20]  Hanspeter Mössenböck,et al.  The taming of the shrew: increasing performance by automatic parameter tuning for java garbage collectors , 2014, ICPE.

[21]  Wayne A. Babich,et al.  Software Configuration Management: Coordination for Team Productivity , 1986 .

[22]  Shivnath Babu,et al.  Tuning Database Configuration Parameters with iTuned , 2009, Proc. VLDB Endow..

[23]  Avishai Wool,et al.  A quantitative study of firewall configuration errors , 2004, Computer.

[24]  Tore Dybå,et al.  A systematic review of effect size in software engineering experiments , 2007, Inf. Softw. Technol..

[25]  Takayuki Osogami,et al.  Finding probably better system configurations quickly , 2006, SIGMETRICS '06/Performance '06.

[26]  Takayuki Osogami,et al.  Optimizing system configurations quickly by guessing at the performance , 2007, SIGMETRICS '07.

[27]  Xiao Ma,et al.  An empirical study on configuration errors in commercial and open source systems , 2011, SOSP.

[28]  Ahmed E. Hassan,et al.  Detecting performance anti-patterns for applications developed using object-relational mapping , 2014, ICSE.

[29]  Mohamed F. Mokbel,et al.  SARD: A statistical approach for ranking database tuning parameters , 2008, 2008 IEEE 24th International Conference on Data Engineering Workshop.

[30]  Wei Zheng,et al.  Automatic configuration of internet services , 2007, EuroSys '07.

[31]  Ahmad Seddighi.  Spring Persistence with Hibernate , 2009 .