Composing, analyzing and validating software models to assess the performability of competing design candidates

In a perfect world, verification and validation of a software design specification would be possible before any code was generated. Indeed, in a perfect world we would know that the implementation was correct because we could trust the class libraries, the development tools, verification tools and simulations, etc. These features would provide the confidence needed to know that all aspects (complexity, logical and timing correctness) of the design were fully satisfied (i.e., everything was right). Right in the sense that we built it right (it is correct with respect to its specification) and it solves the right problem. Unfortunately, it is not a perfect world, and therefore we must strive to continue to refine, develop and validate useful methods and tools for the creation of safe and correct software. This paper considers the analysis of systems expressed using formal notations. We introduce our framework, the modeling cycle, and motivate the need for tool supported rigorous methods. Our framework is about using systematic formal techniques for the creation and composition of software models that can further enable reasoning about high‐assurance systems. We describe several formal modeling techniques within this context (i.e., reliability and availability models, performance and functional models, performability models, etc.). This discussion includes a more precise discourse on stochastic methods (i.e., DTMC and CTMC) and their formulation. In addition, we briefly review the underlying theories and assumptions that are used to solve these models for the measure of interest (i.e., simulation, numerical and analytical techniques). Finally, we present a simple example that employs generalized stochastic Petri nets and illustrates the usefulness of such analysis methods.

[1]  Jean-Claude Laprie,et al.  Modeling computer systems evolutions: non-stationary processes and stochastic Petri nets-application to dependability growth , 1995, Proceedings 6th International Workshop on Petri Nets and Performance Models.

[2]  Hoon Choi,et al.  Sensitivity Analysis of Deterministic and Stochastic Petri Nets , 1993, IEEE/ACM International Symposium on Modeling, Analysis, and Simulation On Computer and Telecommunication Systems.

[3]  Bruno Sericola,et al.  Performability Analysis Using Semi-Markov Reard Processes , 1990, IEEE Trans. Computers.

[4]  Kishor S. Trivedi,et al.  On the Solution of GSPN Reward Models , 1991, Perform. Evaluation.

[6]  S. Leutenegger,et al.  ON THE UTILITY OF THE MULTI-LEVEL ALGORITHM FOR THE SOLUTION OF NEARLY COMPLETELY DECOMPOSABLE MARKOV CHAINS , 1994 .

[7]  Arnold O. Allen Probability, Statistics, and Queueing Theory , 1978 .

[8]  Ernst-Rüdiger Olderog,et al.  TCSP: Theory of Communicating Sequential Processes , 1986, Advances in Petri Nets.

[9]  Kishor S. Trivedi,et al.  Automated Generation and Analysis of Markov Reward Models Using Stochastic Reward Nets , 1993 .

[10]  Giuliana Franceschinis,et al.  On the Computation of Performance Characteristics of Concurrent Programs Using GSPNs , 1994, Perform. Evaluation.

[11]  Giuliana Franceschinis,et al.  Understanding Parallel Program Behavior through Petri Net Models , 1992, J. Parallel Distributed Comput..

[12]  Edmund M. Clarke,et al.  Symbolic Model Checking: 10^20 States and Beyond , 1990, Inf. Comput..

[13]  Kishor S. Trivedi,et al.  The Effect of Detection and Restoration Times for Error Recovery in Communication Networks , 1995, Proceedings of MILCOM '95.

[14]  Gianfranco Ciardo Toward a Definition of Modeling Power for Stochastic Petri Net Models , 1987, PNPM.

[15]  Malathi Veeraraghavan,et al.  Reliability modeling: an overview for system designers , 1991, Computer.

[16]  Graham Horton,et al.  State space construction and steady-state solution of GSPNs on a shared-memory multiprocessor , 1997, Proceedings of the Seventh International Workshop on Petri Nets and Performance Models.

[17]  C. A. R. Hoare,et al.  Communicating sequential processes , 1978, CACM.

[18]  Arnold O. Allen,et al.  Probablity, Statistics and Queueing Theory with Computer Science Applications, Second Edition , 1990, Int. CMG Conference.

[19]  Giovanni Chiola,et al.  A Software Package for the Analysis of Generalized Stochastic Petri Net Models , 1985, PNPM.

[20]  Kishor S. Trivedi,et al.  A methodology for formal expression of hierarchy in model solution , 1993, Proceedings of 5th International Workshop on Petri Nets and Performance Models.

[21]  Krishna M. Kavi,et al.  Specification and analysis of stochastic properties for concurrent systems expressed using csp , 1996 .

[22]  Gianfranco Balbo Performance Evaluation and Concurrent Programming , 1993, MMB.

[23]  Edward D. Lazowska,et al.  Quantitative system performance - computer system analysis using queueing network models , 1983, Int. CMG Conference.

[24]  Michael K. Molloy Performance Analysis Using Stochastic Petri Nets , 1982, IEEE Transactions on Computers.

[25]  Alan D. Lewis Petri net modeling and software safety analysis: methodology for an embedded military application. , 1988 .

[26]  Kishor S. Trivedi,et al.  Reliability and Performability Techniques and Tools: A Survey , 1993, MMB.

[27]  M.A. Qureshi,et al.  The UltraSAN Modeling Environment , 1995, Perform. Evaluation.

[28]  Frederick T. Sheldon Analysis of Real-Time Concurrent System Models based on CSP Using Stochastic Petri Nets , 1998, ESM.

[29]  Edward D. Lazowska,et al.  Quantitative System Performance , 1985, Int. CMG Conference.

[30]  William J. Stewart,et al.  Introduction to the numerical solution of Markov Chains , 1994 .

[31]  Arnold O. Allen,et al.  Probability, statistics and queueing theory - with computer science applications (2. ed.) , 1981, Int. CMG Conference.

[32]  Kishor S. Trivedi,et al.  Reliability Modeling Using SHARPE , 1987, IEEE Transactions on Reliability.

[33]  William H. Sanders,et al.  Reduced Base Model Construction Methods for Stochastic Activity Networks , 1991, IEEE J. Sel. Areas Commun..

[34]  Markus Siegle Beschreibung und Analyse von Markovmodellen mit grossem Zustandsraum , 1995 .

[35]  Gerard J. Holzmann,et al.  The Model Checker SPIN , 1997, IEEE Trans. Software Eng..

[36]  Gunter Bolch,et al.  Performance evaluation of dynamic priority operating systems , 1995, Proceedings 6th International Workshop on Petri Nets and Performance Models.

[37]  Kishor S. Trivedi,et al.  A Decomposition Approach for Stochastic Reward Net Models , 1993, Perform. Evaluation.

[38]  Hoon Choi,et al.  Sensitivity analysis of Markov regenerative stochastic Petri nets , 1993, Proceedings of 5th International Workshop on Petri Nets and Performance Models.

[39]  Harvey M. Deitel,et al.  An introduction to operating systems , 1984 .

[40]  John B. Evans Structures of Discrete Event Simulation: An Introduction to the Engagement Strategy , 1988 .

[41]  Krishna M. Kavi,et al.  Reliability analysis of CSP specifications - A new method using Petri nets , 1995 .

[42]  Kishor S. Trivedi,et al.  Markov and Markov reward model transient analysis: An overview of numerical approaches , 1989 .

[43]  Sung Deok Cha,et al.  Safety analysis using coloured Petri nets , 1996, Proceedings 1996 Asia-Pacific Software Engineering Conference.

[44]  Tadao Murata,et al.  Petri nets: Properties, analysis and applications , 1989, Proc. IEEE.

[45]  Kishor S. Trivedi,et al.  Stochastic Reward Nets for Reliability Prediction , 1996 .

[46]  Raymond A. Paul,et al.  Toward Integrated Methods for High-Assurance Systems (Guest Editors' Introduction) , 1998, Computer.

[47]  Ian Sommerville,et al.  Software engineering (4th ed.) , 1993 .

[48]  Kishor S. Trivedi,et al.  Performance And Reliability Analysis Of Computer Systems (an Example-based Approach Using The Sharpe Software , 1997, IEEE Transactions on Reliability.

[49]  Kishor S. Trivedi,et al.  Sensitivity analysis of reliability and performability measures for multiprocessor systems , 1988, SIGMETRICS '88.

[50]  Marco Ajmone Marsan,et al.  A class of generalised stochastic petri nets for the performance evaluation of multiprocessor systems , 1983, SIGMETRICS '83.

[51]  Kishor S. Trivedi,et al.  Spnp Users Manual Version 4.0 , 1994 .

[52]  Krishna M. Kavi,et al.  Specification of stochastic properties with CSP , 1994, Proceedings of 1994 International Conference on Parallel and Distributed Systems.

[53]  Gerard J. Holzmann,et al.  Design and Validation of Protocols: A Tutorial , 1993, Comput. Networks ISDN Syst..

[54]  David A. McAllester,et al.  Effective Bayesian Inference for Stochastic Programs , 1997, AAAI/IAAI.

[55]  Jeffrey P. Buzen,et al.  Queueing Network Models of Multiprogramming , 1971, Outstanding Dissertations in the Computer Sciences.

[56]  W. J. Gordon,et al.  Closed Queuing Systems with Exponential Servers , 1967, Oper. Res..

[57]  Boudewijn R. Haverkort,et al.  Performance and reliability analysis of computer systems: An example-based approach using the sharpe software package , 1998 .

[58]  Ernst-Rüdiger Olderog,et al.  Operational Petri net semantics for CCSP , 1986, European Workshop on Applications and Theory of Petri Nets.

[59]  A. Hein,et al.  Conjoint simulation-a technique for the combined performance and dependability analysis of large-scale computer systems , 1996, Proceedings of IEEE International Computer Performance and Dependability Symposium.

[60]  Kishor S. Trivedi,et al.  Dependability evaluation through measurements and models , 1994 .

[61]  Kishor S. Trivedi,et al.  A software tool for learning about stochastic models , 1993 .

[62]  Vincent Hodgson,et al.  Nonpreemptive Priorities in Machine Interference , 1967, Oper. Res..

[63]  Stefano Russo,et al.  Software Architecture of the EPOCA Integrated Environment , 1994, Computer Performance Evaluation.

[64]  Holger Hermanns,et al.  Stochastic Process Algebras , 1995 .

[65]  Kishor S. Trivedi,et al.  Analyzing Concurrent and Fault-Tolerant Software Using Stochastic Reward Nets , 1992, J. Parallel Distributed Comput..

[66]  Krishna M. Kavi,et al.  Linking Software Failure Behavior To Specification , 1994, Third Int'l Workshop on Integrating Error Models with Fault Injection.

[67]  Kishor S. Trivedi,et al.  SPNP: stochastic Petri net package , 1989, Proceedings of the Third International Workshop on Petri Nets and Performance Models, PNPM89.

[68]  Graham Horton,et al.  A multi-level solution algorithm for steady-state Markov chains , 1994, SIGMETRICS.

[69]  Gianfranco Balbo,et al.  On the success of stochastic Petri nets , 1995, Proceedings 6th International Workshop on Petri Nets and Performance Models.

[70]  Gunter Bolch,et al.  Leistungsbewertung von Rechensystemen - mittels analytischer Warteschlangenmodelle , 1989, Leitfäden und Monographien der Informatik.

[71]  Philip Heidelberger,et al.  Sensitivity Analysis of Continuous Time Markov Chains Using Uniformization , 1987, Computer Performance and Reliability.

[72]  Kishor S. Trivedi Probability and Statistics with Reliability, Queuing, and Computer Science Applications , 1984 .

[73]  Robin Milner,et al.  Communication and concurrency , 1989, PHI Series in computer science.

[74]  Alain Jean-Marie,et al.  Quantitative evaluation of discrete event systems: Models, performances and techniques , 1993, Proceedings of 5th International Workshop on Petri Nets and Performance Models.

[75]  Susanna Donatelli,et al.  A comparison of performance evaluation process algebra and generalized stochastic Petri nets , 1995, Proceedings 6th International Workshop on Petri Nets and Performance Models.