Collaborative Software Performance Engineering for Enterprise Applications

In the domain of enterprise applications, organizations usually implement third-party standard software components in order to save costs. Hence, application performance monitoring activities constantly produce log entries that are comparable to a certain extent, holding the potential for valuable collaboration across organizational borders. Taking advantage of this fact, we propose a collaborative knowledge base, aimed to support decisions of performance engineering activities, carried out during early design phases of planned enterprise applications. To verify our assumption of crossorganizational comparability, machine learning algorithms were trained on monitoring logs of 18,927 standard application instances productively running at different organizations around the globe. Using random forests, we were able to predict the mean response time for selected standard business transactions with a mean relative error of 23.19 percent. Hence, the approach combines benefits of existing measurement-based and model-based performance prediction techniques, leading to competitive advantages, enabled by interorganizational collaboration.

[1]  Gianluca Bontempi,et al.  A data analysis method for software performance prediction , 2002, Proceedings 2002 Design, Automation and Test in Europe Conference and Exhibition.

[2]  Daniel A. Menascé,et al.  Composing Web Services: A QoS View , 2004, IEEE Internet Comput..

[3]  Antonín Steinhauser,et al.  Utilizing Performance Unit Tests To Increase Performance Awareness , 2015, ICPE.

[4]  Michael Httermann,et al.  DevOps for Developers , 2012 .

[5]  Jens Happe,et al.  The Performance Cockpit Approach: A Framework For Systematic Performance Evaluations , 2010, 2010 36th EUROMICRO Conference on Software Engineering and Advanced Applications.

[6]  Samuel Kounev,et al.  Modeling event-based communication in component-based software architectures for performance predictions , 2013, Software & Systems Modeling.

[7]  Rajkumar Buyya,et al.  Energy Efficient Resource Management in Virtualized Cloud Data Centers , 2010, 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing.

[8]  Will Cappelli Magic Quadrant for Application Performance Monitoring , 2010 .

[9]  Helmut Krcmar,et al.  Generic performance prediction for ERP and SOA applications , 2011, ECIS.

[10]  Kay Wilhelm Capacity Planning for SAP , 2001 .

[11]  Leonid Grinshpan Solving Enterprise Applications Performance Puzzles: Queuing Models to the Rescue , 2012 .

[12]  Samuel Kounev,et al.  Architecture-level software performance abstractions for online performance prediction , 2014, Sci. Comput. Program..

[13]  Ion Stoica,et al.  Ernest: Efficient Performance Prediction for Large-Scale Advanced Analytics , 2016, NSDI.

[14]  Paola Inverardi,et al.  Model-based performance prediction in software development: a survey , 2004, IEEE Transactions on Software Engineering.

[15]  Martin Bichler,et al.  Design science in information systems research , 2006, Wirtschaftsinf..

[16]  David J. Lilja,et al.  Measuring computer performance : A practitioner's guide , 2000 .

[17]  Kay Chen Tan,et al.  On solving multiobjective bin packing problems using evolutionary particle swarm optimization , 2008, Eur. J. Oper. Res..

[18]  Dorina C. Petriu,et al.  The Future of Software Performance Engineering , 2007, Future of Software Engineering (FOSE '07).

[19]  Michael Hüttermann DevOps for Developers , 2012, Apress.

[20]  Gunter Saake,et al.  Predicting performance via automated feature-interaction detection , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[21]  Liming Zhu,et al.  DevOps - A Software Architect's Perspective , 2015, SEI series in software engineering.

[22]  Sven Apel,et al.  Cost-Efficient Sampling for Performance Prediction of Configurable Systems (T) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).