Tools for Performance Evaluation of Computer Systems: Historical Evolution and Perspectives

The development of software tools for performance evaluation and modeling has been an active research area since the early years of computer science. In this paper, we offer a short overview of historical evolution of the field with an emphasis on popular performance modeling techniques such as queuing networks and Petri nets. A review of recent works that provide new perspectives to software tools for performance modeling is presented, followed by a number of ideas on future research directions for the area.

[1]  Harry M. Markowitz,et al.  SIMSCRIPT: A Simulation Programming Language , 1962 .

[2]  Marco Ajmone Marsan,et al.  Modelling with Generalized Stochastic Petri Nets , 1995, PERV.

[3]  K. Mani Chandy,et al.  Open, Closed, and Mixed Networks of Queues with Different Classes of Customers , 1975, JACM.

[4]  Giuseppe Serazzi,et al.  Special issue on tools for computer performance modeling and reliability analysis , 2009, PERV.

[5]  William H. Sanders,et al.  Adversary-driven state-based system security evaluation , 2010, MetriSec '10.

[6]  Leonard Kleinrock,et al.  Queueing Systems: Volume I-Theory , 1975 .

[7]  S. Wittevrongel,et al.  Queueing Systems , 2019, Introduction to Stochastic Processes and Simulation.

[8]  Ramon Puigjaner,et al.  Computer Performance Evaluation , 2000, Lecture Notes in Computer Science.

[9]  Jane Hillston,et al.  Fluid flow approximation of PEPA models , 2005, Second International Conference on the Quantitative Evaluation of Systems (QEST'05).

[10]  Giuseppe Serazzi,et al.  Measurement and Tuning of Computer Systems , 1984, Int. CMG Conference.

[11]  Kishor S. Trivedi,et al.  Non-Markovian Petri nets , 1995, SIGMETRICS '95/PERFORMANCE '95.

[12]  Kurt Jensen,et al.  Coloured Petri nets (2nd ed.): basic concepts, analysis methods and practical use: volume 1 , 1996 .

[13]  Philip Heidelberger,et al.  A spectral method for confidence interval generation and run length control in simulations , 1981, CACM.

[14]  S. Mohan,et al.  Performance Solutions: A Practical Guide to Creating Responsive, Scalable Software [Book Review] , 2003, IEEE Software.

[15]  Gianfranco Ciardo,et al.  Comparison of structural formalisms for modeling large Markov models , 2004, 18th International Parallel and Distributed Processing Symposium, 2004. Proceedings..

[16]  Norman R. Nielsen.  Computer simulation of computer system performance , 1967, ACM '67.

[17]  Connie U. Smith,et al.  Performance solutions: a practical guide to creating responsive , 2001 .

[18]  William J. Stewart,et al.  Introduction to the numerical solution of Markov Chains , 1994 .

[19]  Steffen Becker,et al.  Automatically improve software architecture models for performance, reliability, and cost using evolutionary algorithms , 2010, WOSP/SIPEW '10.

[20]  Günter Haring,et al.  Performance Evaluation: Origins and Directions , 2000, Lecture Notes in Computer Science.

[21]  Jeffrey P. Buzen,et al.  Computational algorithms for closed queueing networks with exponential servers , 1973, Commun. ACM.

[22]  Connie U. Smith,et al.  Performance model interchange format (PMIF 2.0): XML definition and implementation , 2004, First International Conference on the Quantitative Evaluation of Systems, 2004. QEST 2004. Proceedings..

[23]  Heinrich Müller,et al.  Effiziente Methoden der geometrischen Modellierung und der wissenschaftlichen Visualisierung, Dagstuhl Seminar 1997 , 1999, Effiziente Methoden der geometrischen Modellierung und der wissenschaftlichen Visualisierung.

[24]  Ramón Puigjaner,et al.  Performance Model Interchange Format (PMIF 2): A comprehensive approach to Queueing Network Model interoperability , 2010, Perform. Evaluation.

[25]  Peter Buchholz,et al.  Complexity of Memory-Efficient Kronecker Operations with Applications to the Solution of Markov Models , 2000, INFORMS J. Comput..

[26]  George S. Fishman,et al.  Statistical Analysis for Queueing Simulations , 1973 .

[27]  Krzysztof Pawlikowski,et al.  Steady-state simulation of queueing processes: survey of problems and solutions , 1990, CSUR.

[28]  Giovanni Chiola,et al.  Stochastic Well-Formed Colored Nets and Symmetric Modeling Applications , 1993, IEEE Trans. Computers.

[29]  K. Mani Chandy,et al.  Computer Systems Performance Modeling , 1981 .

[30]  Stephen S. Lavenberg,et al.  A Perspective on Queueing Models of Computer Performance , 1989, Perform. Evaluation.

[31]  Gary Carlson.  A User's View of Hardware Performance Monitors, or How to Get More Computer for Your Dollar , 1971, IFIP Congress.

[32]  Peter W. Glynn,et al.  A Linear Programming Algorithm for Computing the Stationary Distribution of Semimartingale Re∞ected Brownian Motion , 2008 .

[33]  Luigi Portinale,et al.  Radyban: A tool for reliability analysis of dynamic fault trees through conversion into dynamic Bayesian networks , 2008, Reliab. Eng. Syst. Saf..

[34]  Connie U. Smith,et al.  Performance Engineering Evaluation of CORBA-based Distributed Systems with SPE*ED , 1998, Computer Performance Evaluation.

[35]  S. Rahman.  Reliability Engineering and System Safety , 2011 .

[36]  Deborah A. Sheetz,et al.  BEST/1 - Design of a tool for computer system capacity planning , 1899, AFIPS National Computer Conference.

[37]  Donald J. Herman,et al.  The use of a computer to evaluate computers , 1964, AFIPS '64 (Spring).

[38]  C. U. Smith,et al.  Performance model interchange format (PMIF 2.0): XML definition and implementation , 2004 .