Software Performance Testing in Virtual Time

We show how traditional unit testing frameworks can be extended to support the simultaneous testing of behaviour and performance, by embedding performance models in mock objects. Because such models are virtual, and therefore execute in virtual time, performance tests can often be performed substantially quicker than when real resources are involved. Performance models also facilitate testing before some or all of a unit»s intended collaborators have been implemented. A key technical challenge is to overcome the impedance mismatch which arises when code that is executing in real time has to communicate with performance models that execute in virtual time. Solutions to this problem naturally facilitate virtual time scaling of both real code and performance models. We also explore potential applications of such time scaling in software performance testing and optimisation.

[1]  Petr Tuma,et al.  Unit Testing Performance in Java Projects: Are We There Yet? , 2017, ICPE.

[2]  Kewen Wang,et al.  Performance Prediction for Apache Spark Platform , 2015, 2015 IEEE 17th International Conference on High Performance Computing and Communications, 2015 IEEE 7th International Symposium on Cyberspace Safety and Security, and 2015 IEEE 12th International Conference on Embedded Software and Systems.

[3]  Steve Freeman,et al.  Endo-testing: unit testing with mock objects , 2001 .

[4]  Andreas S. Andreou,et al.  Empirical evidence and state of practice of software agile teams , 2014, J. Softw. Evol. Process..

[5]  Mauro Iacono,et al.  Performance evaluation of NoSQL big-data applications using multi-formalism models , 2014, Future Gener. Comput. Syst..

[6]  Connie U. Smith,et al.  Performance Engineering of Software Systems , 1990, SIGMETRICS Perform. Evaluation Rev..

[7]  Dorina C. Petriu,et al.  The Future of Software Performance Engineering , 2007, Future of Software Engineering (FOSE '07).

[8]  Emery D. Berger,et al.  Coz: finding code that counts with causal profiling , 2015, USENIX Annual Technical Conference.

[9]  Daniel A. Menascé,et al.  Performance Model Derivation of Operational Systems through Log Analysis , 2016, 2016 IEEE 24th International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS).

[10]  Tony Field,et al.  Software Performance Prediction with a Time Scaling Scheduling Profiler , 2011, 2011 IEEE 19th Annual International Symposium on Modelling, Analysis, and Simulation of Computer and Telecommunication Systems.

[11]  Giuseppe Serazzi,et al.  A Queueing Network Model for Performance Prediction of Apache Cassandra , 2017, VALUETOOLS.